Test a hypothesis about a difference in a continuous outcome variable.
Source:R/hypothesis_evaluation.R
test_mdiff.Rdtest_mdiff is suitable for conducting a testing a hypothesis about the
magnitude of difference between two conditions for a continuous outcome
variable. It can test hypotheses about differences in means or medians for
both independent and paired designs.
Arguments
- estimate
An esci_estimate object generated by an estimate_mdiff_ function
- effect_size
One of 'mean' or 'median'. The effect size selected must be available in the esci_estimate object; medians are only available when the estimate was generated from raw data.
- rope
A two-element vector defining the Region of Practical Equivalence (ROPE). Specify c(0, 0) to test a point null of exactly 0. Specify any two ascending values to test an interval null (e.g. c(-1, 1) to test the hypothesis tha the difference is between -1 and 1).
- rope_units
One of 'raw' (default) or 'sd', specifies the units of the ROPE. If 'sd' is specified, the rope is defined in standard deviation units (e.g. c(-1, 1) is taken as between -1 and 1 standard deviations from 0). When sd is used, the ROPE is converted to raw scores and then the test is conducted on raw scores.
- output_html
TRUE to return results in HTML; FALSE (default) to return standard output
Value
Returns a list with 1-2 data frames
point_null - always returned
test_type - 'Nil hypothesis test', meaning a test against H0 = 0
outcome_variable_name - Name of the outcome variable
effect - Label for the effect being tested
null_words - Express the null in words
confidence - Confidence level, integer (95 for 95%, etc.)
LL - Lower boundary of the confidence% CI for the effect
UL - Upper boundary of the confidence% CI for the effect
CI - Character representation of the CI for the effect
CI_compare - Text description of relation between CI and null
t - If applicable, t value for hypothesis test
df - If applicable, degrees of freedom for hypothesis test
p - If applicable, p value for hypothesis test
p_result - Text representation of p value obtained
null_decision - Text represention of the decision for the null
conclusion - Text representation of conclusion to draw
significant - TRUE/FALSE if significant at alpha = 1-CI
interval_null - returned only if an interval null is specified
test_type - 'Practical significance test', meaning a test against an interval null
outcome_variable_name -
effect - Name of the outcome variable
rope - Test representation of null interval
confidence - Confidence level, integer (95 for 95%, etc.)
CI - Character representation of the CI for the effect
rope_compare - Text description of relation between CI and null interval
p_result - Text representation of p value obtained
conclusion - Text representation of conclusion to draw
significant - TRUE/FALSE if significant at alpha = 1-CI
Details
This function can be passed an esci_estimate object generated by
estimate_mdiff_one(), estimate_mdiff_two(),
estimate_mdiff_paired(), or estimate_mdiff_ind_contrast().
It can test hypotheses about a specific value for the difference (a point null) or about a range of values (an interval null)
Examples
# example code
data("data_penlaptop1")
estimate <- esci::estimate_mdiff_two(
data = data_penlaptop1,
outcome_variable = transcription,
grouping_variable = condition,
switch_comparison_order = TRUE,
assume_equal_variance = TRUE
)
# Test mean difference against point null of 0
esci::test_mdiff(
estimate,
effect_size = "mean"
)
#> $properties
#> $properties$effect_size_name
#> [1] "mean"
#>
#> $properties$alpha
#> [1] 0.05
#>
#> $properties$interval_null
#> [1] FALSE
#>
#> $properties$rope
#> [1] 0 0
#>
#> $properties$rope_units
#> [1] "raw"
#>
#>
#> $point_null
#> test_type outcome_variable_name effect null_words confidence
#> 1 Nil Hypothesis Test transcription Pen ‒ Laptop 0.00 95
#> LL UL CI
#> 1 -8.729915 -2.685265 95% CI [-8.729915, -2.685265]
#> CI_compare t df p p_result
#> 1 The 95% CI does not contain H_0 -3.77382 63 0.0003579282 p < 0.05
#> null_decision conclusion
#> 1 Reject H_0 At α = 0.05, 0.00 is not a plausible value of μ_diff
#> significant
#> 1 TRUE
#>
# Test median difference against point null of 0
# Note that t, df, p return NA because test is completed
# by interval.
esci::test_mdiff(
estimate,
effect_size = "median"
)
#> $properties
#> $properties$effect_size_name
#> [1] "median"
#>
#> $properties$alpha
#> [1] 0.05
#>
#> $properties$interval_null
#> [1] FALSE
#>
#> $properties$rope
#> [1] 0 0
#>
#> $properties$rope_units
#> [1] "raw"
#>
#>
#> $point_null
#> test_type outcome_variable_name effect null_words confidence
#> 1 Nil Hypothesis Test transcription Pen ‒ Laptop 0.00 95
#> LL UL CI
#> 1 -8.085644 -0.3143563 95% CI [-8.085644, -0.3143563]
#> CI_compare t df p p_result null_decision
#> 1 The 95% CI does not contain H_0 NA NA NA p < 0.05 Reject H_0
#> conclusion significant
#> 1 At α = 0.05, 0.00 is not a plausible value of η_diff TRUE
#>
# Test mean difference against interval null of -10 to 10
esci::test_mdiff(
estimate,
effect_size = "mean",
rope = c(-10, 10)
)
#> $properties
#> $properties$effect_size_name
#> [1] "mean"
#>
#> $properties$alpha
#> [1] 0.05
#>
#> $properties$interval_null
#> [1] TRUE
#>
#> $properties$rope
#> [1] -10 10
#>
#> $properties$rope_units
#> [1] "raw"
#>
#>
#> $point_null
#> test_type outcome_variable_name effect null_words confidence
#> 1 Nil Hypothesis Test transcription Pen ‒ Laptop 0.00 95
#> LL UL CI
#> 1 -8.729915 -2.685265 95% CI [-8.729915, -2.685265]
#> CI_compare t df p p_result
#> 1 The 95% CI does not contain H_0 -3.77382 63 0.0003579282 p < 0.05
#> null_decision conclusion
#> 1 Reject H_0 At α = 0.05, 0.00 is not a plausible value of μ_diff
#> significant
#> 1 TRUE
#>
#> $interval_null
#> test_type outcome_variable_name effect
#> 1 Practical significance test transcription Pen ‒ Laptop
#> rope confidence
#> 1 (-10.00, 10.00) 95
#> CI
#> 1 95% CI [-8.729915, -2.685265]\n90% CI [-8.232423, -3.182757]
#> rope_compare p_result conclusion
#> 1 90% CI fully inside H_0 p < 0.05 At α = 0.05, conclude μ_diff is negligible
#> significant
#> 1 TRUE
#>
# Test mean difference against interval null of d (-0.20, 0.20) d = 0.2 is often
# thought of as a small effect, so this test examines if the effect is
# negligible (clearly between negligble and small), substantive (clearly more
# than small), or unclear. The d boundaries provided are converted to raw scores
# and then the CI of the observed effect is compared to the raw-score boundaries
esci::test_mdiff(
estimate,
effect_size = "mean",
rope = c(-0.2, 0.2),
rope_units = "sd"
)
#> $properties
#> $properties$effect_size_name
#> [1] "mean"
#>
#> $properties$alpha
#> [1] 0.05
#>
#> $properties$interval_null
#> [1] TRUE
#>
#> $properties$rope
#> [1] -0.2 0.2
#>
#> $properties$rope_units
#> [1] "sd"
#>
#>
#> $point_null
#> test_type outcome_variable_name effect null_words confidence
#> 1 Nil Hypothesis Test transcription Pen ‒ Laptop 0.00 95
#> LL UL CI
#> 1 -8.729915 -2.685265 95% CI [-8.729915, -2.685265]
#> CI_compare t df p p_result
#> 1 The 95% CI does not contain H_0 -3.77382 63 0.0003579282 p < 0.05
#> null_decision conclusion
#> 1 Reject H_0 At α = 0.05, 0.00 is not a plausible value of μ_diff
#> significant
#> 1 TRUE
#>
#> $interval_null
#> test_type outcome_variable_name effect
#> 1 Practical significance test transcription Pen ‒ Laptop
#> rope confidence
#> 1 (-1.21805, 1.21805) 95
#> CI
#> 1 95% CI [-8.729915, -2.685265]\n90% CI [-8.232423, -3.182757]
#> rope_compare p_result conclusion
#> 1 95% CI fully outside H_0 p < 0.05 At α = 0.05, conclude μ_diff is substantive
#> significant
#> 1 TRUE
#>