Week 4: Visualizing Associations and Models

Emorie D Beck

Review

Review

  • Last week, we talked about how to visualize proportions and probabilities, which often assumes the conjunction of categorical and count data
  • This week, we’ll focus not on relative differences across categories but rather on how measures co-vary
  • There’s a ton of ways to do this statistically, which is not the focus of this course or this workshop
  • Instead, we’ll focus on visualizations that work across cases (e.g., correlations) and model types (lots of generalizable support in R for different modeling packages!)

Final Project Proposals

Final Project Proposals

  • Due at 11:59 PM PST on February 12, 2025
  • 1-2 page (single spaced) proposal
    • Short background (why do you / we care?)
    • Research question(s; only the question[s] you are focusing on)
    • Short method (what are the data? Which are you using?)
    • Visualization plan (short summary of your proposed visualization)
    • Challenges and barriers (what do you struggle with this visualization; are there specific barriers?)

Final Project Proposals

Visualization Plan

Can be any combination of the following:

  • Text description
    • clearly describe axes/scales, fills/colors, panels, etc.
    • describe affordances you will build into the visualization to aid understanding
    • describe why you chose this visualization
  • Draft visualization (digital)
    • rough visualization of some or all of data
  • Draft visualization (drawn)
    • some roughly drawn visualization showing what you are hoping to do
    • particularly helpful if you foresee barriers or don’t know how to do it
    • can also draw these on a plot of yours from the past, etc.

Final Project Proposals

  • The goal of this proposal is to:
    • Provide you with support to build visualizations you may be nervous about
    • Anticipate barriers and provide you tools
    • Set you up to create a visualization that you are proud of

Final Project Proposals

  • What is okay?
    • Introducing a new kind of visualization to your area of research
    • Offering improvements to “standard practice” visualizations in your area
    • Setting up a procedure for a kind of visualization you’ve long wanted / needed to figure out
    • Really anything that is both useful to you and displays some sort of mastery of course content

Part 1 Visualizing Associations Among Quantitative Variables

The Data

We’re going to work with a subsample of data from multiple studies. These are simulated versions of data from my dissertation simulated to mirror the underlying structure of each study, including their drawbacks.

load(url("https://github.com/emoriebeck/psc290-data-viz-2022/blob/main/04-week4-associations/04-data/week4-data.RData?raw=true"))
pred_data
# A tibble: 5,021 × 25
   study  o_value p_year SID     p_value     age gender grsWages parEdu race 
   <chr>  <fct>    <dbl> <chr>     <dbl>   <dbl> <fct>     <dbl> <fct>  <fct>
 1 Study1 0         2005 61215      6.67 -29.9   1         1.02  2      0    
 2 Study1 0         2005 184965     0    -22.9   0         1.14  2      0    
 3 Study1 0         2005 488251    10     -3.92  1         0.717 1      0    
 4 Study1 0         2005 650779     7.22 -25.9   1         0.644 3      0    
 5 Study1 0         2005 969691     7.22  -0.925 1         0.812 2      0    
 6 Study1 0         2005 986687     6.11  14.1   0         1.76  2      0    
 7 Study1 0         2005 1054011    5.56   8.08  0         1.34  1      0    
 8 Study1 0         2005 1372251    7.78   5.08  1         0.842 1      0    
 9 Study1 0         2005 1496703    6.11 -23.9   0         1.42  2      0    
10 Study1 0         2005 1897887    2.78  38.1   1         0.725 2      0    
# ℹ 5,011 more rows
# ℹ 15 more variables: physhlthevnt <fct>, SRhealth <dbl>, smokes <fct>,
#   alcohol <fct>, exercise <dbl>, BMI <dbl>, parDivorce <fct>, PhysFunc <fct>,
#   religion <fct>, education <fct>, married <fct>, numKids <dbl>,
#   parOccPrstg <dbl>, reliability <dbl>, predInt <dbl>

Scatterplots

  • Scatterplots are pretty ubiquitous
  • From a data visualization standpoint, this makes sense
  • Scatterplots
    • show raw data
    • are common enough that little visualization literacy is needed
    • allow for lots of summaries to be placed atop them
    • this is why they are our entry point for today

Scatterplots - Basics

  • First, let’s peek at our data
  • These data are based on some of my dissertation (Beck & Jackson, 2022) work that takes data from lots of people across lots of samples and examines personality trait predictors of a bunch of outcomes across all possible combinations of covariates (i.e. specification curve analysis or multiverse analysis)
pred_data
# A tibble: 5,021 × 25
   study  o_value p_year SID     p_value     age gender grsWages parEdu race 
   <chr>  <fct>    <dbl> <chr>     <dbl>   <dbl> <fct>     <dbl> <fct>  <fct>
 1 Study1 0         2005 61215      6.67 -29.9   1         1.02  2      0    
 2 Study1 0         2005 184965     0    -22.9   0         1.14  2      0    
 3 Study1 0         2005 488251    10     -3.92  1         0.717 1      0    
 4 Study1 0         2005 650779     7.22 -25.9   1         0.644 3      0    
 5 Study1 0         2005 969691     7.22  -0.925 1         0.812 2      0    
 6 Study1 0         2005 986687     6.11  14.1   0         1.76  2      0    
 7 Study1 0         2005 1054011    5.56   8.08  0         1.34  1      0    
 8 Study1 0         2005 1372251    7.78   5.08  1         0.842 1      0    
 9 Study1 0         2005 1496703    6.11 -23.9   0         1.42  2      0    
10 Study1 0         2005 1897887    2.78  38.1   1         0.725 2      0    
# ℹ 5,011 more rows
# ℹ 15 more variables: physhlthevnt <fct>, SRhealth <dbl>, smokes <fct>,
#   alcohol <fct>, exercise <dbl>, BMI <dbl>, parDivorce <fct>, PhysFunc <fct>,
#   religion <fct>, education <fct>, married <fct>, numKids <dbl>,
#   parOccPrstg <dbl>, reliability <dbl>, predInt <dbl>

Scatterplots - Basics

  • We’re going to start by looking at associations between Conscientiousness (p_value) and self-rated helath (SRhealth)
  • Both have been rescaled based on the Percentage of Maximum Possible (Cohen et al., 1999) with a range of 0-10 to help model convergence
pred_data %>% 
  select(study, SID, p_value, SRhealth)
# A tibble: 5,021 × 4
   study  SID     p_value SRhealth
   <chr>  <chr>     <dbl>    <dbl>
 1 Study1 61215      6.67     9.23
 2 Study1 184965     0        7.5 
 3 Study1 488251    10        6.43
 4 Study1 650779     7.22     6.92
 5 Study1 969691     7.22     5.77
 6 Study1 986687     6.11     6.84
 7 Study1 1054011    5.56     6   
 8 Study1 1372251    7.78     3.62
 9 Study1 1496703    6.11     6.59
10 Study1 1897887    2.78     5.62
# ℹ 5,011 more rows

Scatterplots - Basics

Let’s look at a basic scatterplot of the association:

pred_data %>% 
  select(study, SID, p_value, SRhealth) %>%
  ggplot(aes(x = p_value, y = SRhealth)) + 
    geom_point(shape = 21, fill = "grey80", color = "black", size = 2) + 
    labs(
      x = "Agreeableness (POMP; 0-10)"
      , y = "Self-Rated Health (POMP; 0-10)"
    ) + 
    theme_classic()

Scatterplots - Basics

Let’s look at a basic scatterplot of the association with a trend line:

pred_data %>% 
  select(study, SID, p_value, SRhealth) %>%
  ggplot(aes(x = p_value, y = SRhealth)) + 
    geom_point(shape = 21, fill = "grey80", color = "black", size = 2) + 
    geom_smooth(method = "lm", size = 3, se = F) + 
    labs(
      x = "Conscientiousness (POMP; 0-10)"
      , y = "Self-Rated Health (POMP; 0-10)"
    ) + 
    theme_classic()

Scatterplots - Basics

But we have multiple studies, so we need to separate those out using facet_wrap()

pred_data %>% 
  select(study, SID, p_value, SRhealth) %>%
  filter(!is.na(SRhealth)) %>%
  ggplot(aes(x = p_value, y = SRhealth)) + 
    geom_point(shape = 21, fill = "grey80", color = "black", size = 2) + 
    scale_fill_manual(values = c("grey80", "seagreen4")) + 
    facet_wrap(~study) +
    labs(
      x = "Conscientiousness (POMP; 0-10)"
      , y = "Self-Rated Health (POMP; 0-10)"
    ) + 
    theme_classic()

Scatterplots - Basics

Let’s add a trend line again and change the alpha of the points to make them stand out a bit less:

pred_data %>% 
  select(study, SID, p_value, SRhealth) %>%
  filter(!is.na(SRhealth)) %>%
  ggplot(aes(x = p_value, y = SRhealth)) + 
    geom_point(shape = 21, fill = "grey80", color = "black", size = 2, alpha = .25) + 
    geom_smooth(method = "lm", size = 2, se = F) + 
    scale_fill_manual(values = c("grey80", "seagreen4")) + 
    facet_wrap(~study) +
    labs(
      x = "Conscientiousness (POMP; 0-10)"
      , y = "Self-Rated Health (POMP; 0-10)"
    ) + 
    theme_classic()

Scatterplots - Basics

We usually want to have some interval estimate around any average or other measure of central tendency, so we’ll set se = T in geom_smooth()

pred_data %>% 
  select(study, SID, p_value, SRhealth) %>%
  filter(!is.na(SRhealth)) %>%
  ggplot(aes(x = p_value, y = SRhealth)) + 
    geom_point(shape = 21, fill = "grey80", color = "black", size = 2, alpha = .25) + 
    geom_smooth(method = "lm", size = 1.5, se = T, color = "black") + 
    scale_fill_manual(values = c("grey80", "seagreen4")) + 
    facet_wrap(~study) +
    labs(
      x = "Conscientiousness (POMP; 0-10)"
      , y = "Self-Rated Health (POMP; 0-10)"
      , title = "Conscientiousness -- Self-Rated Health Associations Across Samples"
    ) + 
    theme_classic()

Correlations and Correlelograms

  • Understanding associations is always important, but perhaps never more so than when we do descriptives
  • My hot take is that zero-order correlation matrices should always be included in papers
    • Someone’s meta-analysis will thank you
  • If you’re dumping correlations in supplementary materials, then tables are fine
  • But you (and your brain) will thank yourself if you use heat maps or correlelograms to visualize the correlations
    • (Remember how quickly and preattentively we perceive color and size?)
  • There are R packages for this (e.g., corrplot), but where’s the fun in that?

Correlations and Correlelograms

All right, let’s estimate some correlation matrices for each sample:

r_data <- pred_data %>%
  select(
    study, p_value, age, gender, SRhealth, smokes, exercise
    , BMI, education, parEdu, mortality = o_value
    ) %>%
  mutate_if(is.factor, ~as.numeric(as.character(.))) %>%
  group_by(study) %>%
  nest() %>%
  ungroup() %>%
  mutate(r = map(data, ~cor(., use = "pairwise")))
r_data
# A tibble: 6 × 3
  study  data                  r              
  <chr>  <list>                <list>         
1 Study1 <tibble [831 × 10]>   <dbl [10 × 10]>
2 Study2 <tibble [1,000 × 10]> <dbl [10 × 10]>
3 Study3 <tibble [1,000 × 10]> <dbl [10 × 10]>
4 Study4 <tibble [574 × 10]>   <dbl [10 × 10]>
5 Study5 <tibble [616 × 10]>   <dbl [10 × 10]>
6 Study6 <tibble [1,000 × 10]> <dbl [10 × 10]>

Correlations and Correlelograms

The thing is that we know ggplot doesn’t like wide form data, which is what cor() produces

r_data$r[[1]]
               p_value          age       gender    SRhealth       smokes
p_value    1.000000000 -0.005224085  0.053627861  0.15917525 -0.069013463
age       -0.005224085  1.000000000 -0.057243245 -0.22438335 -0.078788619
gender     0.053627861 -0.057243245  1.000000000 -0.03182278  0.022275557
SRhealth   0.159175251 -0.224383351 -0.031822781  1.00000000 -0.129241536
smokes    -0.069013463 -0.078788619  0.022275557 -0.12924154  1.000000000
exercise   0.048576025 -0.361768736  0.061659017  0.34546038 -0.155018841
BMI       -0.019741798  0.036151816  0.012217132 -0.09340105 -0.037713371
education  0.001465775 -0.173399716 -0.001603648  0.11008540 -0.096936630
parEdu     0.019871078 -0.374733606  0.055468171  0.08273023  0.005215303
mortality -0.089637524  0.627069166 -0.092109448 -0.31142292  0.035759332
             exercise         BMI    education       parEdu   mortality
p_value    0.04857602 -0.01974180  0.001465775  0.019871078 -0.08963752
age       -0.36176874  0.03615182 -0.173399716 -0.374733606  0.62706917
gender     0.06165902  0.01221713 -0.001603648  0.055468171 -0.09210945
SRhealth   0.34546038 -0.09340105  0.110085399  0.082730234 -0.31142292
smokes    -0.15501884 -0.03771337 -0.096936630  0.005215303  0.03575933
exercise   1.00000000 -0.06217297  0.210204022  0.176766791 -0.32138385
BMI       -0.06217297  1.00000000 -0.048914825 -0.075000576  0.01643219
education  0.21020402 -0.04891483  1.000000000  0.232321970 -0.17215791
parEdu     0.17676679 -0.07500058  0.232321970  1.000000000 -0.18796244
mortality -0.32138385  0.01643219 -0.172157913 -0.187962436  1.00000000

Correlations and Correlelograms

  • So we need to reshape it in long form
  • We’re going to use a function so we only have to write the code once and can apply it to all the samples
  • Here’s what we’ll do:
    • remove the lower triangle and the diagonal of the correlation matrix
    • make matrix a data frame
    • make the row names of the matrix a column
    • make the columns long
    • factor them to retain order
r_reshape_fun <- function(r){
  coln <- colnames(r)
  # remove lower tri and diagonal
  r[lower.tri(r, diag = T)] <- NA
  r %>% data.frame() %>%
    rownames_to_column("V1") %>%
    pivot_longer(
      cols = -V1
      , values_to = "r"
      , names_to = "V2"
    ) %>%
    mutate(V1 = factor(V1, coln)
           , V2 = factor(V2, rev(coln)))
}

r_data <- r_data %>%
  mutate(r = map(r, r_reshape_fun))
r_data$r[[1]]
# A tibble: 100 × 3
   V1      V2               r
   <fct>   <fct>        <dbl>
 1 p_value p_value   NA      
 2 p_value age       -0.00522
 3 p_value gender     0.0536 
 4 p_value SRhealth   0.159  
 5 p_value smokes    -0.0690 
 6 p_value exercise   0.0486 
 7 p_value BMI       -0.0197 
 8 p_value education  0.00147
 9 p_value parEdu     0.0199 
10 p_value mortality -0.0896 
# ℹ 90 more rows

Correlations and Correlelograms

Heat Map Time!

This is, technically, a heat map, but I think we can do better!

r_data$r[[1]] %>%
  ggplot(aes(
    x = V1
    , y = V2
    , fill = r
  )) + 
  geom_raster() + 
  theme_minimal()

Correlations and Correlelograms

Heat Map Time! Colors

Let’s add some intuitive colors using scale_fill_gradient2()

r_data$r[[1]] %>%
  ggplot(aes(x = V1, y = V2, fill = r)) + 
  geom_raster() + 
  scale_fill_gradient2(
    limits = c(-1,1)
    , breaks = c(-1, -.5, 0, .5, 1)
    , low = "blue"
    , high = "red"
    , mid = "white"
    , na.value = "white"
    ) + 
  theme_minimal()

Correlations and Correlelograms

Heat Map Time! Labels

Do we need axis labels? Not really – let’s remove them and add fill and title labels:

r_data$r[[1]] %>%
  ggplot(aes(x = V1, y = V2, fill = r)) + 
  geom_raster() + 
  scale_fill_gradient2(limits = c(-1,1)
    , breaks = c(-1, -.5, 0, .5, 1)
    , low = "blue", high = "red"
    , mid = "white", na.value = "white") + 
  labs(
    x = NULL
    , y = NULL
    , fill = "Zero-Order Correlation"
    , title = "Zero-Order Correlations Among Variables in Sample 1"
    ) + 
  theme_minimal()

Correlations and Correlelograms

Heat Map Time! Theme Elements

Let’s fix the theme elements. So close!

r_data$r[[1]] %>%
  ggplot(aes(x = V1, y = V2, fill = r)) + 
  geom_raster() + 
  scale_fill_gradient2(limits = c(-1,1)
    , breaks = c(-1, -.5, 0, .5, 1)
    , low = "blue", high = "red"
    , mid = "white", na.value = "white") + 
  labs(
    x = NULL
    , y = NULL
    , fill = "Zero-Order Correlation"
    , title = "Zero-Order Correlations Among Variables"
    , subtitle = "Sample 1"
    ) + 
  theme_classic() + 
  theme(
    legend.position = "bottom"
    , axis.text = element_text(face = "bold")
    , axis.text.x = element_text(angle = 45, hjust = 1)
    , plot.title = element_text(face = "bold", hjust = .5)
    , plot.subtitle = element_text(face = "italic", hjust = .5)
    , panel.background = element_rect(color = "black", linewidth = 1)
  )

Correlations and Correlelograms

Heat Map Time! Finishing Touches!

And add text to the correlations using geom_text():

r_data$r[[1]] %>%
  ggplot(aes(x = V1, y = V2, fill = r)) + 
  geom_raster() + 
  geom_text(aes(label = round(r, 2))) + 
  scale_fill_gradient2(limits = c(-1,1)
    , breaks = c(-1, -.5, 0, .5, 1)
    , low = "blue", high = "red"
    , mid = "white", na.value = "white") + 
  labs(
    x = NULL
    , y = NULL
    , fill = "Zero-Order Correlation"
    , title = "Zero-Order Correlations Among Variables"
    , subtitle = "Sample 1"
    ) + 
  theme_classic() + 
  theme(
    legend.position = "bottom"
    , axis.text = element_text(face = "bold")
    , axis.text.x = element_text(angle = 45, hjust = 1)
    , plot.title = element_text(face = "bold", hjust = .5)
    , plot.subtitle = element_text(face = "italic", hjust = .5)
    , panel.background = element_rect(color = "black", size = 1)
  )

Correlations and Correlelograms

Correlelogram

A correlelogram is basically a heat map that uses size in addition to color. To practice your skills, you’re going to create a correllogram on your own. Using the following base-code, build a publication worthy plot.

Some hints and suggestions: - You’ll use geom_point() - Here’s all the ggplot shapes. I suggest checking out shape 21. - Be thoughtful about your legends - Be thoughtful about your axis labels and titles - Hint: you can borrow some code from the heat map

r_data$r[[1]] %>%
  ggplot(aes(x = V1, y = V2, color = r, size = abs(r))) + 
  geom_point() + 
  theme_classic()

Correlations and Correlelograms

Correlelogram

r_data$r[[1]] %>%
  ggplot(aes(x = V1, y = V2, fill = r, size = abs(r))) + 
  geom_point(shape = 21) + 
  scale_fill_gradient2(limits = c(-1,1)
    , breaks = c(-1, -.5, 0, .5, 1)
    , low = "blue", high = "red"
    , mid = "white", na.value = "white") + 
  scale_size_continuous(range = c(3,14)) + 
  labs(
    x = NULL
    , y = NULL
    , fill = "Zero-Order\nCorrelation"
    , title = "Zero-Order Correlations Among Variables"
    , subtitle = "Sample 1"
    ) + 
  guides(size = "none") + 
  theme_classic() + 
  theme(
    legend.position = "bottom"
    , axis.text = element_text(face = "bold")
    , axis.text.x = element_text(angle = 45, hjust = 1)
    , plot.title = element_text(face = "bold", hjust = .5)
    , plot.subtitle = element_text(face = "italic", hjust = .5)
    , panel.background = element_rect(color = "black", size = 1)
  )

Part 2 Visualizing Associations, Parameters, and Predictions from Models

Part 2 Visualizing Associations, Parameters, and Predictions from Models

  • The goal of data visualization is to tell a story that tables, words, etc. either can’t or can’t do simply
  • Data visualizations aims to clarify complex patterns in data
  • Thus far, we’ve mostly focused on building models from raw data or descriptives of raw data
  • But in most research, we lean on inferential statistics and hypothesis testing (frequent or Bayesian) to tell our story
  • So next, we’ll talk about how to use data visualization to tell stories with models

Part 2 Visualizing Associations, Parameters, and Predictions from Models

Telling Stories with Models

  • The reality is that there is no generalizable way to do this
  • So we will focus on models for which we are interested in specific parameters and/or parameterized our questions
  • Why? These have some shared functions across lots of packages in R
  • For models that don’t, that’s a data cleaning problem, not a visualization problem

Part 2 Visualizing Associations, Parameters, and Predictions from Models

  • Let’s start with a basic model and predict later all-cause mortality from Conscientiousness in Sample 1.
  • The basic form of the model is:

\[ logit(\frac{\pi}{1-\pi}) = b_0 + b_1*C_{ij} + \epsilon_{ij} \]

ds1 <- pred_data %>% filter(study == "Study1")
m1 <- glm(
  o_value ~ p_value
  , data = ds1
  , family = binomial(link = "logit")
  )
summary(m1)

Call:
glm(formula = o_value ~ p_value, family = binomial(link = "logit"), 
    data = ds1)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)  
(Intercept)  0.23213    0.25920   0.896   0.3705  
p_value     -0.09349    0.03632  -2.574   0.0101 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 1117.4  on 830  degrees of freedom
Residual deviance: 1110.7  on 829  degrees of freedom
AIC: 1114.7

Number of Fisher Scoring iterations: 4

Part 2 Visualizing Associations, Parameters, and Predictions from Models

  • Models and other objects in R are stored in lists or list-like objects
  • We can explore these lots of ways, but one good one is with str() 
str(m1)
List of 30
 $ coefficients     : Named num [1:2] 0.2321 -0.0935
  ..- attr(*, "names")= chr [1:2] "(Intercept)" "p_value"
 $ residuals        : Named num [1:831] -1.68 -2.26 -1.5 -1.64 -1.64 ...
  ..- attr(*, "names")= chr [1:831] "1" "2" "3" "4" ...
 $ fitted.values    : Named num [1:831] 0.403 0.558 0.331 0.391 0.391 ...
  ..- attr(*, "names")= chr [1:831] "1" "2" "3" "4" ...
 $ effects          : Named num [1:831] 5.755 -2.574 -0.729 -0.78 -0.78 ...
  ..- attr(*, "names")= chr [1:831] "(Intercept)" "p_value" "" "" ...
 $ R                : num [1:2, 1:2] -14.1 0 -96.5 27.5
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : chr [1:2] "(Intercept)" "p_value"
  .. ..$ : chr [1:2] "(Intercept)" "p_value"
 $ rank             : int 2
 $ qr               :List of 5
  ..$ qr   : num [1:831, 1:2] -14.0555 0.0353 0.0335 0.0347 0.0347 ...
  .. ..- attr(*, "dimnames")=List of 2
  .. .. ..$ : chr [1:831] "1" "2" "3" "4" ...
  .. .. ..$ : chr [1:2] "(Intercept)" "p_value"
  ..$ rank : int 2
  ..$ qraux: num [1:2] 1.03 1.12
  ..$ pivot: int [1:2] 1 2
  ..$ tol  : num 1e-11
  ..- attr(*, "class")= chr "qr"
 $ family           :List of 13
  ..$ family    : chr "binomial"
  ..$ link      : chr "logit"
  ..$ linkfun   :function (mu)  
  ..$ linkinv   :function (eta)  
  ..$ variance  :function (mu)  
  ..$ dev.resids:function (y, mu, wt)  
  ..$ aic       :function (y, n, mu, wt, dev)  
  ..$ mu.eta    :function (eta)  
  ..$ initialize: language {     if (NCOL(y) == 1) { ...
  ..$ validmu   :function (mu)  
  ..$ valideta  :function (eta)  
  ..$ simulate  :function (object, nsim)  
  ..$ dispersion: num 1
  ..- attr(*, "class")= chr "family"
 $ linear.predictors: Named num [1:831] -0.391 0.232 -0.703 -0.443 -0.443 ...
  ..- attr(*, "names")= chr [1:831] "1" "2" "3" "4" ...
 $ deviance         : num 1111
 $ aic              : num 1115
 $ null.deviance    : num 1117
 $ iter             : int 4
 $ weights          : Named num [1:831] 0.241 0.247 0.222 0.238 0.238 ...
  ..- attr(*, "names")= chr [1:831] "1" "2" "3" "4" ...
 $ prior.weights    : Named num [1:831] 1 1 1 1 1 1 1 1 1 1 ...
  ..- attr(*, "names")= chr [1:831] "1" "2" "3" "4" ...
 $ df.residual      : int 829
 $ df.null          : int 830
 $ y                : Named num [1:831] 0 0 0 0 0 0 0 0 0 0 ...
  ..- attr(*, "names")= chr [1:831] "1" "2" "3" "4" ...
 $ converged        : logi TRUE
 $ boundary         : logi FALSE
 $ model            :'data.frame':  831 obs. of  2 variables:
  ..$ o_value: Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
  ..$ p_value: num [1:831] 6.67 0 10 7.22 7.22 ...
  ..- attr(*, "terms")=Classes 'terms', 'formula'  language o_value ~ p_value
  .. .. ..- attr(*, "variables")= language list(o_value, p_value)
  .. .. ..- attr(*, "factors")= int [1:2, 1] 0 1
  .. .. .. ..- attr(*, "dimnames")=List of 2
  .. .. .. .. ..$ : chr [1:2] "o_value" "p_value"
  .. .. .. .. ..$ : chr "p_value"
  .. .. ..- attr(*, "term.labels")= chr "p_value"
  .. .. ..- attr(*, "order")= int 1
  .. .. ..- attr(*, "intercept")= int 1
  .. .. ..- attr(*, "response")= int 1
  .. .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv> 
  .. .. ..- attr(*, "predvars")= language list(o_value, p_value)
  .. .. ..- attr(*, "dataClasses")= Named chr [1:2] "factor" "numeric"
  .. .. .. ..- attr(*, "names")= chr [1:2] "o_value" "p_value"
 $ call             : language glm(formula = o_value ~ p_value, family = binomial(link = "logit"), data = ds1)
 $ formula          :Class 'formula'  language o_value ~ p_value
  .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv> 
 $ terms            :Classes 'terms', 'formula'  language o_value ~ p_value
  .. ..- attr(*, "variables")= language list(o_value, p_value)
  .. ..- attr(*, "factors")= int [1:2, 1] 0 1
  .. .. ..- attr(*, "dimnames")=List of 2
  .. .. .. ..$ : chr [1:2] "o_value" "p_value"
  .. .. .. ..$ : chr "p_value"
  .. ..- attr(*, "term.labels")= chr "p_value"
  .. ..- attr(*, "order")= int 1
  .. ..- attr(*, "intercept")= int 1
  .. ..- attr(*, "response")= int 1
  .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv> 
  .. ..- attr(*, "predvars")= language list(o_value, p_value)
  .. ..- attr(*, "dataClasses")= Named chr [1:2] "factor" "numeric"
  .. .. ..- attr(*, "names")= chr [1:2] "o_value" "p_value"
 $ data             : tibble [831 × 25] (S3: tbl_df/tbl/data.frame)
  ..$ study       : chr [1:831] "Study1" "Study1" "Study1" "Study1" ...
  ..$ o_value     : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
  ..$ p_year      : num [1:831] 2005 2005 2005 2005 2005 ...
  ..$ SID         : chr [1:831] "61215" "184965" "488251" "650779" ...
  ..$ p_value     : num [1:831] 6.67 0 10 7.22 7.22 ...
  ..$ age         : num [1:831] -29.925 -22.925 -3.925 -25.925 -0.925 ...
  ..$ gender      : Factor w/ 2 levels "0","1": 2 1 2 2 2 1 1 2 1 2 ...
  ..$ grsWages    : num [1:831] 1.021 1.139 0.717 0.644 0.812 ...
  ..$ parEdu      : Factor w/ 3 levels "1","2","3": 2 2 1 3 2 2 1 1 2 2 ...
  ..$ race        : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...
  ..$ physhlthevnt: Factor w/ 2 levels "0","1": 1 2 2 2 2 2 2 2 2 2 ...
  ..$ SRhealth    : num [1:831] 9.23 7.5 6.43 6.92 5.77 ...
  ..$ smokes      : Factor w/ 2 levels "0","1": 1 2 1 1 1 2 2 1 2 1 ...
  ..$ alcohol     : Factor w/ 2 levels "0","1": NA NA NA NA NA NA NA NA NA NA ...
  ..$ exercise    : num [1:831] 3.75 6.25 5 10 4.5 ...
  ..$ BMI         : num [1:831] 2.02 1.84 1.67 1.89 3.29 ...
  ..$ parDivorce  : Factor w/ 2 levels "0","1": 2 1 1 1 1 1 1 1 1 1 ...
  ..$ PhysFunc    : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...
  ..$ religion    : Factor w/ 3 levels "0","1","2": 1 1 2 2 2 1 1 2 2 2 ...
  ..$ education   : Factor w/ 3 levels "0","1","2": 1 1 1 1 1 1 1 1 1 1 ...
  ..$ married     : Factor w/ 2 levels "0","1": 1 1 2 1 2 2 2 2 1 1 ...
  ..$ numKids     : num [1:831] 0 0 0.588 0 1.765 ...
  ..$ parOccPrstg : num [1:831] 3.67 5.41 1.21 4.66 4.99 ...
  ..$ reliability : num [1:831] 0.511 0.511 0.511 0.511 0.511 ...
  ..$ predInt     : num [1:831] 13 13 13 13 13 13 13 13 13 13 ...
 $ offset           : NULL
 $ control          :List of 3
  ..$ epsilon: num 1e-08
  ..$ maxit  : num 25
  ..$ trace  : logi FALSE
 $ method           : chr "glm.fit"
 $ contrasts        : NULL
 $ xlevels          : Named list()
 - attr(*, "class")= chr [1:2] "glm" "lm"

Models + broom

  • The broom package is great for working with models (and the broom.mixed add-on makes it even better)
  • We’re going to talk about how three its functions can be used for / improve data visualization:
    • tidy()
    • glance()
    • augment()

Models + broom::tidy()

  • Outside of dplyr/tidyr, tidy() is a close contender with purrr::map() functions as my most used function
  • Why?
    • When you run a model, base R provides the summary(), coef(), etc. to extract various components of the model
    • But these aren’t data.frames, which are core input to a lot of other R functions across packages
    • tidy() provides a data frame with core model coefficients, inferential tests, etc. that be easily matched and merged across models, etc.

Models + broom::tidy()

  • But with logistic regression with a logit link, we are left with coefficents that have to be interpreted in log odds, which realistically, almost no one can do
  • So we have to “undo” the log, which you may remember can done by exponentiating the natural log (ln)
  • But we can directly exponentiate from the summary function because it’s the wrong class of object
  • We could just exponentiate the coefficients from the coef() function, but this still leaves us with the need to extract estimates of precision, like standard errors, confidence intervals, and more.
coef(m1)
(Intercept)     p_value 
  0.2321341  -0.0934916

Models + broom::tidy()

Enter broom::tidy()!

tidy(m1)
# A tibble: 2 × 5
  term        estimate std.error statistic p.value
  <chr>          <dbl>     <dbl>     <dbl>   <dbl>
1 (Intercept)   0.232     0.259      0.896  0.370 
2 p_value      -0.0935    0.0363    -2.57   0.0101

Models + broom::tidy()

Enter broom::tidy()!
Even better, we can easily get confidence intervals

tidy(m1, conf.int = T)
# A tibble: 2 × 7
  term        estimate std.error statistic p.value conf.low conf.high
  <chr>          <dbl>     <dbl>     <dbl>   <dbl>    <dbl>     <dbl>
1 (Intercept)   0.232     0.259      0.896  0.370    -0.276    0.741 
2 p_value      -0.0935    0.0363    -2.57   0.0101   -0.165   -0.0225

Models + broom::tidy()

  • We’d rarely make a plot for just two parameters unless we’re doing it across groups or samples
  • Watch! Let’s make a nested data frame that will hold
    • All the data for each sample
    • A model for each sample
    • The tidy() data frame of the parameter estimates for each sample
tidy_ci <- function(m) tidy(m, conf.int = T)

nested_m <- pred_data %>%
  group_by(study) %>%
  nest() %>%
  ungroup() %>%
  mutate(
    m = map(data
            , ~glm(
              o_value ~p_value
              , data = .
              , family = binomial(link = "logit")
              )
            )
    , tidy = map(m, tidy_ci)
  )
nested_m
# A tibble: 6 × 4
  study  data                  m      tidy            
  <chr>  <list>                <list> <list>          
1 Study1 <tibble [831 × 24]>   <glm>  <tibble [2 × 7]>
2 Study2 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]>
3 Study3 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]>
4 Study4 <tibble [574 × 24]>   <glm>  <tibble [2 × 7]>
5 Study5 <tibble [616 × 24]>   <glm>  <tibble [2 × 7]>
6 Study6 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]>

Models + broom::tidy()

Now, we’ll drop the data and m columns that we don’t need and unnest() our tidy() data frames

nested_m %>%
  select(study, tidy) %>%
  unnest(tidy)
# A tibble: 12 × 8
   study  term        estimate std.error statistic    p.value conf.low conf.high
   <chr>  <chr>          <dbl>     <dbl>     <dbl>      <dbl>    <dbl>     <dbl>
 1 Study1 (Intercept)   0.232     0.259      0.896 0.370        -0.276    0.741 
 2 Study1 p_value      -0.0935    0.0363    -2.57  0.0101       -0.165   -0.0225
 3 Study2 (Intercept)   1.42      0.310      4.60  0.00000425    0.823    2.04  
 4 Study2 p_value      -0.185     0.0392    -4.71  0.00000246   -0.262   -0.108 
 5 Study3 (Intercept)   1.32      0.328      4.03  0.0000565     0.685    1.97  
 6 Study3 p_value      -0.166     0.0404    -4.11  0.0000390    -0.246   -0.0877
 7 Study4 (Intercept)  -1.62      0.464     -3.49  0.000482     -2.57    -0.741 
 8 Study4 p_value      -0.0581    0.0898    -0.647 0.518        -0.232    0.121 
 9 Study5 (Intercept)  -1.17      0.496     -2.36  0.0183       -2.16    -0.215 
10 Study5 p_value      -0.0391    0.0655    -0.597 0.550        -0.167    0.0905
11 Study6 (Intercept)   0.277     0.336      0.826 0.409        -0.380    0.937 
12 Study6 p_value      -0.0367    0.0437    -0.841 0.400        -0.123    0.0488

Models + broom::tidy()

Now these parameters from multiple models, we may want to plot!

nested_m %>%
  select(study, tidy) %>%
  unnest(tidy) %>%
  mutate_at(vars(estimate, conf.low, conf.high), exp) %>%
  ggplot(
    aes(y = study, x = estimate)
  ) + 
    geom_errorbar(
      aes(xmin = conf.low, xmax = conf.high)
      , position = position_dodge(width = .9)
      , width = .1
      ) + 
    geom_point() + 
    theme_classic()

Models + broom::tidy()

Almost, but we have two parameters for each model (Intercept and p_value), so let’s split those in a facet:

nested_m %>%
  select(study, tidy) %>%
  unnest(tidy) %>%
  mutate_at(vars(estimate, conf.low, conf.high), exp) %>%
  ggplot(
    aes(y = study, x = estimate)
  ) + 
    geom_errorbar(
      aes(xmin = conf.low, xmax = conf.high)
      , position = position_dodge(width = .9)
      , width = .1
      ) + 
    geom_point() + 
    facet_grid(~term) + 
    theme_classic()

Models + broom::tidy()

We’ve got some work to do to make this an intuitive figure. Let’s:

  • Add a dashed line at 1 (odd ratio of 1 is a null effect)
  • Let the scales vary for different terms
  • Make the points bigger
  • Fix the titles on the plot and axis titles
  • Add some color
  • Fiddle with themes to make it prettier

Models + broom::tidy()

nested_m %>%
  select(study, tidy) %>%
  unnest(tidy) %>%
  mutate_at(vars(estimate, conf.low, conf.high), exp) %>%
  ggplot(
    aes(y = study, x = estimate, fill = study)
  ) + 
    geom_vline(aes(xintercept = 1), linetype = "dashed") + 
    geom_errorbar(
      aes(xmin = conf.low, xmax = conf.high)
      , position = position_dodge(width = .9)
      , width = .1
      ) + 
    geom_point(size = 3, shape = 22) + 
    labs(
      x = "Estimate (CI) in OR"
      , y = NULL
      , title = "Conscientiousness was associated with mortality 50% of samples"
      , subtitle = "Samples with lower mortality risk overall had fewer significant associations"
      ) + 
    facet_grid(~term, scales = "free") + 
    theme_classic() + 
    theme(
      legend.position = "none"
      , axis.text = element_text(face = "bold", size = rel(1.1))
      , axis.title = element_text(face = "bold", size = rel(1.2))
      , axis.line = element_blank()
      , strip.text = element_text(face = "bold", size = rel(1.1), color = "white")
      , strip.background = element_rect(fill = "black")
      , plot.title = element_text(face = "bold", size = rel(1.1), hjust = .5)
      , plot.subtitle = element_text(face = "italic", size = rel(1.1))
      , panel.border = element_rect(color = "black", fill = NA, size = 1)
    )

Models + broom::tidy()

  • This isn’t perfect. But we’re going to come back to this kind of plot when we talk about “piecing plots together.”
  • Personally, I would:
    • Add text with Est. (CI) and N for each sample in the figure
    • Build both of these separately in order to order by effect size
    • Then put them back together and re-add the title

Models + broom::glance()`

  • When we run models, we need to care about more that just point and interval estimates
  • Often we are interested in comparing models, checking diagnostics, etc.
  • Again, all of these are embedded (mostly), in the model objects
  • The glance() function brings some of these important ones into a single object
  • Here’s what it gives us for our logistic regression model
glance(m1)
# A tibble: 1 × 8
  null.deviance df.null logLik   AIC   BIC deviance df.residual  nobs
          <dbl>   <int>  <dbl> <dbl> <dbl>    <dbl>       <int> <int>
1         1117.     830  -555. 1115. 1124.    1111.         829   831

Models + broom::glance()`

  • Let’s also look for a linear model and t-test (also a linear model), which may be more familiar for many of you:
m2 <- lm(SRhealth ~ age, data = ds1)
glance(m2)
# A tibble: 1 × 12
  r.squared adj.r.squared sigma statistic  p.value    df logLik   AIC   BIC
      <dbl>         <dbl> <dbl>     <dbl>    <dbl> <dbl>  <dbl> <dbl> <dbl>
1    0.0503        0.0492  1.60      44.0 6.06e-11     1 -1571. 3148. 3162.
# ℹ 3 more variables: deviance <dbl>, df.residual <int>, nobs <int>
m3 <- t.test(SRhealth ~ gender, data = ds1)
glance(m3)
# A tibble: 1 × 10
  estimate estimate1 estimate2 statistic p.value parameter conf.low conf.high
     <dbl>     <dbl>     <dbl>     <dbl>   <dbl>     <dbl>    <dbl>     <dbl>
1    0.105      6.62      6.52     0.915   0.360      819.   -0.120     0.329
# ℹ 2 more variables: method <chr>, alternative <chr>

Models + broom::glance()`

As before, we can do this with lots of models to compare across samples:

nested_m 
# A tibble: 6 × 4
  study  data                  m      tidy            
  <chr>  <list>                <list> <list>          
1 Study1 <tibble [831 × 24]>   <glm>  <tibble [2 × 7]>
2 Study2 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]>
3 Study3 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]>
4 Study4 <tibble [574 × 24]>   <glm>  <tibble [2 × 7]>
5 Study5 <tibble [616 × 24]>   <glm>  <tibble [2 × 7]>
6 Study6 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]>

Models + broom::glance()`

As before, we can do this with lots of models to compare across samples:

nested_m <- nested_m %>%
  mutate(glance = map(m, glance))
nested_m
# A tibble: 6 × 5
  study  data                  m      tidy             glance          
  <chr>  <list>                <list> <list>           <list>          
1 Study1 <tibble [831 × 24]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
2 Study2 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
3 Study3 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
4 Study4 <tibble [574 × 24]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
5 Study5 <tibble [616 × 24]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
6 Study6 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>

Models + broom::glance()`

As before, we can do this with lots of models to compare across samples
Now unnesting:

nested_m %>%
  select(study, glance) %>%
  unnest(glance)
# A tibble: 6 × 9
  study  null.deviance df.null logLik   AIC   BIC deviance df.residual  nobs
  <chr>          <dbl>   <int>  <dbl> <dbl> <dbl>    <dbl>       <int> <int>
1 Study1         1117.     830  -555. 1115. 1124.    1111.         829   831
2 Study2         1386.     999  -682. 1367. 1377.    1363.         998  1000
3 Study3         1386.     999  -684. 1373. 1383.    1369.         998  1000
4 Study4          441.     573  -220.  445.  454.     441.         572   574
5 Study5          596.     615  -298.  600.  608.     596.         614   616
6 Study6         1386.     999  -693. 1390. 1399.    1386.         998  1000

Models + broom::glance()`

Realistically, this is the kind of info we table, but we can also merge it with info from tidy:

nested_m %>%
  select(-data, -m) %>%
  unnest(tidy) %>% 
  unnest(glance) %>%
  mutate_if(is.numeric, ~round(., 2))
study term estimate std.error statistic p.value conf.low conf.high null.deviance df.null logLik AIC BIC deviance df.residual nobs
Study1 (Intercept) 0.23 0.26 0.90 0.37 -0.28 0.74 1117.40 830 -555.36 1114.73 1124.17 1110.73 829 831
Study1 p_value -0.09 0.04 -2.57 0.01 -0.17 -0.02 1117.40 830 -555.36 1114.73 1124.17 1110.73 829 831
Study2 (Intercept) 1.42 0.31 4.60 0.00 0.82 2.04 1386.29 999 -681.62 1367.23 1377.05 1363.23 998 1000
Study2 p_value -0.18 0.04 -4.71 0.00 -0.26 -0.11 1386.29 999 -681.62 1367.23 1377.05 1363.23 998 1000
Study3 (Intercept) 1.32 0.33 4.03 0.00 0.68 1.97 1386.29 999 -684.41 1372.82 1382.64 1368.82 998 1000
Study3 p_value -0.17 0.04 -4.11 0.00 -0.25 -0.09 1386.29 999 -684.41 1372.82 1382.64 1368.82 998 1000
Study4 (Intercept) -1.62 0.46 -3.49 0.00 -2.57 -0.74 441.21 573 -220.40 444.80 453.50 440.80 572 574
Study4 p_value -0.06 0.09 -0.65 0.52 -0.23 0.12 441.21 573 -220.40 444.80 453.50 440.80 572 574
Study5 (Intercept) -1.17 0.50 -2.36 0.02 -2.16 -0.21 596.00 615 -297.82 599.64 608.49 595.64 614 616
Study5 p_value -0.04 0.07 -0.60 0.55 -0.17 0.09 596.00 615 -297.82 599.64 608.49 595.64 614 616
Study6 (Intercept) 0.28 0.34 0.83 0.41 -0.38 0.94 1386.29 999 -692.79 1389.59 1399.40 1385.59 998 1000
Study6 p_value -0.04 0.04 -0.84 0.40 -0.12 0.05 1386.29 999 -692.79 1389.59 1399.40 1385.59 998 1000

Models + broom::augment()

  • Diagnostics are not just summary statistics!
  • We care a lot about prediction, too
    • Residuals both tell us unexplained variance (i.e. how observed data deviate from model predictions) and how appropriate our model was
    • Model predictions and prediction intervals tell us about how our model is doing across levels our variables

Let’s keep working with our nested data frame. Remember, it looks like this:

nested_m
# A tibble: 6 × 5
  study  data                  m      tidy             glance          
  <chr>  <list>                <list> <list>           <list>          
1 Study1 <tibble [831 × 24]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
2 Study2 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
3 Study3 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
4 Study4 <tibble [574 × 24]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
5 Study5 <tibble [616 × 24]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
6 Study6 <tibble [1,000 × 24]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>

Models + broom::augment()

  • augment() let’s us add (augment) the raw data we feed the model based on the fitted model
  • Notice we now have more columns
nested_m <- nested_m %>%
  mutate(data = map2(m, data, augment, se_fit = T))
nested_m
# A tibble: 6 × 5
  study  data                  m      tidy             glance          
  <chr>  <list>                <list> <list>           <list>          
1 Study1 <tibble [831 × 31]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
2 Study2 <tibble [1,000 × 31]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
3 Study3 <tibble [1,000 × 31]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
4 Study4 <tibble [574 × 31]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
5 Study5 <tibble [616 × 31]>   <glm>  <tibble [2 × 7]> <tibble [1 × 8]>
6 Study6 <tibble [1,000 × 31]> <glm>  <tibble [2 × 7]> <tibble [1 × 8]>

Models + broom::augment()

  • Here’s the columns we used along with the additional columns with a glm:
    • .fitted: fitted / predicted value
    • .se.fit: standard error
    • .resid: observed - fitted
    • .std.resd: standardized residuals
    • .sigma: estimated residual SD when this obs is dropped from model
    • cooksd: Cooks distance (is this an outlier?)
nested_m$data[[1]] %>%
  select(o_value, SID, p_value, .fitted:.cooksd)
# A tibble: 831 × 9
   o_value SID     p_value .fitted .se.fit .resid    .hat .sigma  .cooksd
   <fct>   <chr>     <dbl>   <dbl>   <dbl>  <dbl>   <dbl>  <dbl>    <dbl>
 1 0       61215      6.67 -0.391   0.0715 -1.02  0.00123   1.16 0.000417
 2 0       184965     0     0.232   0.259  -1.28  0.0166    1.16 0.0108  
 3 0       488251    10    -0.703   0.134  -0.897 0.00400   1.16 0.000998
 4 0       650779     7.22 -0.443   0.0723 -0.996 0.00125   1.16 0.000401
 5 0       969691     7.22 -0.443   0.0723 -0.996 0.00125   1.16 0.000401
 6 0       986687     6.11 -0.339   0.0762 -1.04  0.00141   1.16 0.000504
 7 0       1054011    5.56 -0.287   0.0855 -1.06  0.00179   1.16 0.000674
 8 0       1372251    7.78 -0.495   0.0785 -0.976 0.00145   1.16 0.000444
 9 0       1496703    6.11 -0.339   0.0762 -1.04  0.00141   1.16 0.000504
10 0       1897887    2.78 -0.0276  0.165  -1.17  0.00677   1.16 0.00334 
# ℹ 821 more rows

Models + broom::augment()

For the most part, many of the checks with glm’s and lm’s are the same. But it’s a bit easier to wrap your head around lm(), so let’s switch to that:

nested_lm <- pred_data %>%
  select(study, SID, p_value, age, SRhealth) %>%
  drop_na() %>%
  group_by(study) %>%
  nest() %>%
  ungroup() %>%
  mutate(m = map(data, ~lm(SRhealth ~ p_value + age, data = .))
         , tidy = map(m, tidy_ci)
         , glance = map(m, glance)
         , data = map2(m, data, augment, se_fit = T, interval = "confidence"))
nested_lm
# A tibble: 6 × 5
  study  data                  m      tidy             glance           
  <chr>  <list>                <list> <list>           <list>           
1 Study1 <tibble [831 × 13]>   <lm>   <tibble [3 × 7]> <tibble [1 × 12]>
2 Study2 <tibble [996 × 13]>   <lm>   <tibble [3 × 7]> <tibble [1 × 12]>
3 Study3 <tibble [1,000 × 13]> <lm>   <tibble [3 × 7]> <tibble [1 × 12]>
4 Study4 <tibble [574 × 13]>   <lm>   <tibble [3 × 7]> <tibble [1 × 12]>
5 Study5 <tibble [616 × 13]>   <lm>   <tibble [3 × 7]> <tibble [1 × 12]>
6 Study6 <tibble [1,000 × 13]> <lm>   <tibble [3 × 7]> <tibble [1 × 12]>

Models + broom::augment()

  • Here’s the columns we used along with the additional columns with an lm:
    • .fitted: fitted / predicted value
    • .se.fit: standard error
    • .lower: lower bound of the confidence/prediction interval
    • .upper: upper bound of the confidence/prediction interval
    • .resid: observed - fitted
    • .std.resd: standardized residuals
    • .sigma: estimated residual SD when this obs is dropped from model
    • cooksd: Cooks distance (is this an outlier?)
nested_lm$data[[1]]
SID p_value age SRhealth .fitted .lower .upper .se.fit .resid .hat .sigma .cooksd .std.resid
61215 6.67 -29.92 9.23 7.21 6.98 7.43 0.11 2.03 0.01 1.58 0.00 1.28
184965 0.00 -22.92 7.50 6.20 5.77 6.63 0.22 1.30 0.02 1.58 0.00 0.83
488251 10.00 -3.92 6.43 7.20 6.99 7.41 0.11 -0.78 0.00 1.58 0.00 -0.49
650779 7.22 -25.92 6.92 7.21 7.00 7.42 0.11 -0.29 0.00 1.58 0.00 -0.18
969691 7.22 -0.92 5.77 6.78 6.66 6.91 0.06 -1.02 0.00 1.58 0.00 -0.64
986687 6.11 14.08 6.84 6.38 6.26 6.50 0.06 0.46 0.00 1.58 0.00 0.29
1054011 5.56 8.08 6.00 6.41 6.28 6.54 0.07 -0.41 0.00 1.58 0.00 -0.26
1372251 7.78 5.08 3.62 6.76 6.64 6.88 0.06 -3.14 0.00 1.58 0.00 -1.98
1496703 6.11 -23.92 6.59 7.03 6.83 7.23 0.10 -0.44 0.00 1.58 0.00 -0.28
1897887 2.78 38.08 5.62 5.53 5.24 5.82 0.15 0.08 0.01 1.58 0.00 0.05
2157663 5.00 -18.92 8.65 6.80 6.59 7.00 0.11 1.86 0.00 1.58 0.00 1.18
2190291 6.11 -2.92 6.85 6.67 6.54 6.80 0.07 0.17 0.00 1.58 0.00 0.11
2652019 8.89 -29.92 9.23 7.50 7.25 7.75 0.13 1.73 0.01 1.58 0.00 1.10
2814531 10.00 27.08 6.62 6.68 6.46 6.89 0.11 -0.06 0.00 1.58 0.00 -0.04
2818611 3.89 16.08 6.15 6.05 5.85 6.25 0.10 0.10 0.00 1.58 0.00 0.06
4739684 7.78 -4.92 6.15 6.93 6.79 7.06 0.07 -0.77 0.00 1.58 0.00 -0.49
4781775 5.56 -15.92 4.54 6.82 6.64 7.00 0.09 -2.28 0.00 1.58 0.00 -1.44
4832767 10.00 32.08 7.31 6.59 6.36 6.82 0.12 0.72 0.01 1.58 0.00 0.45
5038964 7.78 -21.92 9.23 7.22 7.02 7.41 0.10 2.01 0.00 1.58 0.00 1.27
5127284 7.22 -10.92 7.21 6.96 6.81 7.10 0.08 0.26 0.00 1.58 0.00 0.16
5848124 3.33 -27.92 4.62 6.73 6.44 7.02 0.15 -2.11 0.01 1.58 0.01 -1.34
6160884 10.00 7.08 7.69 7.02 6.82 7.22 0.10 0.68 0.00 1.58 0.00 0.43
6549775 5.00 -26.92 9.23 6.93 6.70 7.17 0.12 2.30 0.01 1.58 0.00 1.45
8445764 5.56 -13.92 5.77 6.79 6.61 6.96 0.09 -1.02 0.00 1.58 0.00 -0.64
10485684 7.78 -18.92 9.23 7.16 6.98 7.35 0.09 2.07 0.00 1.58 0.00 1.31
11934084 7.78 -16.92 4.62 7.13 6.95 7.31 0.09 -2.52 0.00 1.58 0.00 -1.59
11974884 6.11 -22.92 6.92 7.01 6.81 7.21 0.10 -0.09 0.00 1.58 0.00 -0.06
12818084 6.11 -15.92 6.92 6.89 6.72 7.07 0.09 0.03 0.00 1.58 0.00 0.02
12831684 5.56 -18.92 2.31 6.87 6.68 7.06 0.10 -4.56 0.00 1.58 0.01 -2.89
13913004 8.89 -0.92 6.92 7.01 6.84 7.17 0.08 -0.08 0.00 1.58 0.00 -0.05
19080922 5.00 -26.92 4.62 6.93 6.70 7.17 0.12 -2.32 0.01 1.58 0.00 -1.47
19978482 6.67 -15.92 7.21 6.97 6.80 7.13 0.09 0.24 0.00 1.58 0.00 0.15
27560442 7.22 49.08 7.42 5.93 5.71 6.16 0.12 1.49 0.01 1.58 0.00 0.94
28254122 1.67 -25.92 8.08 6.47 6.12 6.83 0.18 1.60 0.01 1.58 0.00 1.02
28777682 6.67 -22.92 6.92 7.09 6.89 7.28 0.10 -0.16 0.00 1.58 0.00 -0.10
31409282 1.67 9.08 9.23 5.88 5.57 6.19 0.16 3.35 0.01 1.58 0.02 2.13
35421322 6.67 -15.92 8.46 6.97 6.80 7.13 0.09 1.49 0.00 1.58 0.00 0.95
35904082 7.22 -14.92 6.15 7.02 6.86 7.19 0.08 -0.87 0.00 1.58 0.00 -0.55
40086082 5.00 -27.92 7.31 6.95 6.71 7.19 0.12 0.36 0.01 1.58 0.00 0.23
40092922 6.11 -29.92 6.92 7.13 6.90 7.36 0.12 -0.21 0.01 1.58 0.00 -0.13
40419362 6.11 -25.92 8.31 7.06 6.85 7.28 0.11 1.24 0.00 1.58 0.00 0.79
41840442 7.22 -6.92 8.85 6.89 6.75 7.02 0.07 1.96 0.00 1.58 0.00 1.24
42309682 9.44 6.08 7.91 6.96 6.78 7.14 0.09 0.95 0.00 1.58 0.00 0.60
42948842 8.33 3.08 6.76 6.86 6.73 7.00 0.07 -0.11 0.00 1.58 0.00 -0.07
55447322 6.67 -27.92 9.81 7.17 6.95 7.39 0.11 2.64 0.00 1.58 0.00 1.67
55576482 10.00 -14.92 7.85 7.39 7.16 7.63 0.12 0.46 0.01 1.58 0.00 0.29
61261282 6.67 -26.92 8.46 7.15 6.94 7.37 0.11 1.31 0.00 1.58 0.00 0.83
62124842 8.33 3.08 6.69 6.86 6.73 7.00 0.07 -0.17 0.00 1.58 0.00 -0.11
64164842 6.67 4.08 6.63 6.63 6.51 6.74 0.06 0.01 0.00 1.58 0.00 0.01
64919682 6.67 41.08 6.69 6.00 5.80 6.19 0.10 0.70 0.00 1.58 0.00 0.44
65076042 7.78 11.08 6.92 6.65 6.54 6.77 0.06 0.27 0.00 1.58 0.00 0.17
65545242 5.56 2.08 7.85 6.51 6.38 6.65 0.07 1.33 0.00 1.58 0.00 0.84
67680602 6.11 -27.92 6.15 7.10 6.88 7.32 0.11 -0.94 0.01 1.58 0.00 -0.60
68295125 2.22 28.08 7.23 5.63 5.33 5.92 0.15 1.60 0.01 1.58 0.00 1.02
68433169 7.78 12.08 8.23 6.64 6.52 6.75 0.06 1.59 0.00 1.58 0.00 1.01
68441325 4.44 16.08 7.38 6.13 5.95 6.30 0.09 1.26 0.00 1.58 0.00 0.80
68571205 7.22 -0.92 6.92 6.78 6.66 6.91 0.06 0.14 0.00 1.58 0.00 0.09
68629009 8.33 -3.92 8.62 6.98 6.83 7.13 0.08 1.63 0.00 1.58 0.00 1.03
68942485 6.67 24.08 7.23 6.28 6.15 6.42 0.07 0.95 0.00 1.58 0.00 0.60
69338925 8.89 -0.92 8.62 7.01 6.84 7.17 0.08 1.61 0.00 1.58 0.00 1.02
69436913 5.56 -28.92 7.50 7.04 6.81 7.27 0.12 0.46 0.01 1.58 0.00 0.29
69657169 8.89 -5.92 7.12 7.09 6.92 7.26 0.09 0.02 0.00 1.58 0.00 0.02
69706125 10.00 -0.92 8.82 7.15 6.95 7.36 0.11 1.67 0.00 1.58 0.00 1.05
69831253 4.44 -28.92 8.08 6.89 6.63 7.15 0.13 1.18 0.01 1.58 0.00 0.75
69865929 8.33 -5.92 6.92 7.02 6.86 7.17 0.08 -0.09 0.00 1.58 0.00 -0.06
69909445 6.67 5.08 6.54 6.61 6.50 6.72 0.06 -0.07 0.00 1.58 0.00 -0.04
70127049 3.89 31.08 6.43 5.80 5.57 6.02 0.12 0.63 0.01 1.58 0.00 0.40
70216125 6.67 5.08 9.23 6.61 6.50 6.72 0.06 2.62 0.00 1.58 0.00 1.66
70483365 8.89 11.08 6.85 6.80 6.65 6.95 0.08 0.05 0.00 1.58 0.00 0.03
70622765 6.67 21.08 5.85 6.34 6.21 6.46 0.06 -0.49 0.00 1.58 0.00 -0.31
71047085 7.22 -20.92 7.95 7.13 6.94 7.31 0.10 0.82 0.00 1.58 0.00 0.52
71148405 8.89 -5.92 7.79 7.09 6.92 7.26 0.09 0.70 0.00 1.58 0.00 0.44
71434005 5.56 -16.92 6.35 6.84 6.65 7.02 0.10 -0.49 0.00 1.58 0.00 -0.31
72039885 6.11 38.08 7.46 5.97 5.79 6.16 0.09 1.49 0.00 1.58 0.00 0.94
81612929 6.67 6.08 7.75 6.59 6.48 6.70 0.06 1.16 0.00 1.58 0.00 0.73
81889685 7.22 -6.92 5.96 6.89 6.75 7.02 0.07 -0.93 0.00 1.58 0.00 -0.58
82042689 7.78 9.08 6.92 6.69 6.57 6.81 0.06 0.24 0.00 1.58 0.00 0.15
82241925 8.33 17.08 6.10 6.62 6.49 6.76 0.07 -0.53 0.00 1.58 0.00 -0.33
82335085 7.22 24.08 8.08 6.36 6.23 6.49 0.07 1.72 0.00 1.58 0.00 1.09
88939933 9.44 -20.92 7.69 7.42 7.19 7.65 0.12 0.27 0.01 1.58 0.00 0.17
88984805 5.56 -10.92 2.69 6.73 6.57 6.90 0.08 -4.04 0.00 1.58 0.01 -2.56
88984873 6.11 -28.92 5.77 7.11 6.89 7.34 0.11 -1.35 0.01 1.58 0.00 -0.85
89001805 8.89 -5.92 8.57 7.09 6.92 7.26 0.09 1.48 0.00 1.58 0.00 0.94
89096329 8.89 -14.92 9.23 7.24 7.05 7.44 0.10 1.99 0.00 1.58 0.00 1.26
89099725 5.00 -18.92 4.81 6.80 6.59 7.00 0.11 -1.99 0.00 1.58 0.00 -1.26
89271769 5.56 -14.92 6.54 6.80 6.62 6.98 0.09 -0.26 0.00 1.58 0.00 -0.17
95214285 7.78 26.08 6.23 6.40 6.25 6.54 0.07 -0.17 0.00 1.58 0.00 -0.11
95328529 10.00 13.08 5.77 6.91 6.71 7.12 0.10 -1.14 0.00 1.58 0.00 -0.72
95756925 2.78 5.08 2.54 6.09 5.84 6.35 0.13 -3.56 0.01 1.58 0.01 -2.25
95905845 7.22 -19.92 6.46 7.11 6.93 7.29 0.09 -0.65 0.00 1.58 0.00 -0.41
95970445 10.00 -6.92 7.85 7.25 7.04 7.47 0.11 0.59 0.00 1.58 0.00 0.37
96077885 8.89 -5.92 6.23 7.09 6.92 7.26 0.09 -0.86 0.00 1.58 0.00 -0.54
136065965 10.00 -2.92 8.24 7.19 6.98 7.40 0.11 1.06 0.00 1.58 0.00 0.67
136102005 6.67 1.08 6.15 6.68 6.56 6.79 0.06 -0.52 0.00 1.58 0.00 -0.33
136162525 10.00 38.08 7.23 6.49 6.24 6.73 0.13 0.74 0.01 1.58 0.00 0.47
136176129 6.67 11.08 8.15 6.51 6.40 6.62 0.06 1.65 0.00 1.58 0.00 1.04
136205369 4.17 14.08 7.15 6.12 5.94 6.31 0.10 1.03 0.00 1.58 0.00 0.65
136208089 8.33 3.08 8.77 6.86 6.73 7.00 0.07 1.91 0.00 1.58 0.00 1.20
136301929 5.56 14.08 5.54 6.31 6.17 6.44 0.07 -0.77 0.00 1.58 0.00 -0.49
136631053 6.67 -29.92 4.62 7.21 6.98 7.43 0.11 -2.59 0.01 1.58 0.00 -1.64
136634449 5.56 19.08 7.85 6.22 6.08 6.36 0.07 1.62 0.00 1.58 0.00 1.03
136738485 10.00 29.08 8.31 6.64 6.42 6.86 0.11 1.67 0.01 1.58 0.00 1.06
136875857 9.44 -21.92 8.37 7.44 7.20 7.67 0.12 0.93 0.01 1.58 0.00 0.59
137173005 6.67 4.08 4.69 6.63 6.51 6.74 0.06 -1.93 0.00 1.58 0.00 -1.22
137268205 8.89 17.08 7.38 6.70 6.54 6.86 0.08 0.69 0.00 1.58 0.00 0.43
137321245 6.11 12.08 8.69 6.42 6.30 6.53 0.06 2.28 0.00 1.58 0.00 1.44
137697289 5.00 5.08 8.54 6.39 6.24 6.54 0.08 2.15 0.00 1.58 0.00 1.36
137818325 5.00 -15.92 5.69 6.75 6.55 6.94 0.10 -1.05 0.00 1.58 0.00 -0.67
137922369 7.22 -11.92 7.42 6.97 6.82 7.12 0.08 0.45 0.00 1.58 0.00 0.28
138273925 8.33 -10.92 9.14 7.10 6.94 7.27 0.09 2.04 0.00 1.58 0.00 1.29
139070213 10.00 16.08 2.31 6.86 6.66 7.07 0.10 -4.56 0.00 1.58 0.01 -2.88
139176285 6.67 -17.92 7.99 7.00 6.83 7.18 0.09 0.99 0.00 1.58 0.00 0.62
139204845 7.22 16.08 7.31 6.49 6.38 6.61 0.06 0.81 0.00 1.58 0.00 0.51
139217085 6.67 -3.92 6.54 6.76 6.63 6.89 0.07 -0.22 0.00 1.58 0.00 -0.14
139281685 6.11 -19.92 7.97 6.96 6.77 7.15 0.10 1.01 0.00 1.58 0.00 0.64
139621005 6.67 -4.92 8.00 6.78 6.65 6.91 0.07 1.22 0.00 1.58 0.00 0.77
139621009 8.33 -5.92 7.23 7.02 6.86 7.17 0.08 0.21 0.00 1.58 0.00 0.14
150375893 6.11 -29.92 9.23 7.13 6.90 7.36 0.12 2.10 0.01 1.58 0.00 1.33
156758369 10.00 23.08 7.58 6.74 6.53 6.96 0.11 0.84 0.00 1.58 0.00 0.53
163227885 6.11 -0.92 6.46 6.64 6.51 6.77 0.07 -0.18 0.00 1.58 0.00 -0.11
163442085 8.89 12.08 3.46 6.78 6.63 6.94 0.08 -3.32 0.00 1.58 0.00 -2.10
163564485 5.00 -9.92 4.38 6.64 6.46 6.82 0.09 -2.26 0.00 1.58 0.00 -1.43
163986765 3.89 21.08 4.62 5.97 5.76 6.18 0.11 -1.35 0.00 1.58 0.00 -0.86
164155409 8.89 10.08 9.23 6.82 6.67 6.97 0.08 2.41 0.00 1.58 0.00 1.53
164172405 5.00 -9.92 7.62 6.64 6.46 6.82 0.09 0.97 0.00 1.58 0.00 0.61
164260805 8.89 -5.92 6.23 7.09 6.92 7.26 0.09 -0.86 0.00 1.58 0.00 -0.54
204012245 6.67 17.08 7.38 6.40 6.29 6.52 0.06 0.98 0.00 1.58 0.00 0.62
204204005 6.67 15.08 3.23 6.44 6.33 6.55 0.06 -3.21 0.00 1.58 0.00 -2.03
204450169 8.89 21.08 8.08 6.63 6.47 6.79 0.08 1.45 0.00 1.58 0.00 0.91
204450173 10.00 -6.92 8.46 7.25 7.04 7.47 0.11 1.21 0.00 1.58 0.00 0.76
204522925 7.78 18.08 6.62 6.53 6.41 6.66 0.06 0.08 0.00 1.58 0.00 0.05
204552853 8.89 -8.92 6.92 7.14 6.96 7.32 0.09 -0.22 0.00 1.58 0.00 -0.14
204586165 2.78 37.08 7.00 5.55 5.26 5.84 0.15 1.45 0.01 1.58 0.00 0.92
204833689 7.22 -27.92 7.31 7.24 7.03 7.46 0.11 0.06 0.00 1.58 0.00 0.04
204945213 8.33 -18.92 7.85 7.24 7.04 7.43 0.10 0.61 0.00 1.58 0.00 0.38
205085969 7.78 14.08 7.31 6.60 6.48 6.72 0.06 0.71 0.00 1.58 0.00 0.45
205104337 7.22 -23.92 8.24 7.18 6.98 7.38 0.10 1.07 0.00 1.58 0.00 0.67
205359325 10.00 0.08 8.49 7.14 6.93 7.34 0.10 1.35 0.00 1.58 0.00 0.86
205551773 5.56 -22.92 6.35 6.94 6.73 7.15 0.11 -0.59 0.00 1.58 0.00 -0.38
205695245 5.00 34.08 7.58 5.89 5.70 6.09 0.10 1.69 0.00 1.58 0.00 1.07
205712929 8.89 -28.92 8.08 7.48 7.24 7.73 0.12 0.59 0.01 1.58 0.00 0.38
205946165 6.67 -1.92 7.69 6.73 6.61 6.85 0.06 0.96 0.00 1.58 0.00 0.61
206106653 10.00 -28.92 6.92 7.63 7.35 7.91 0.14 -0.71 0.01 1.58 0.00 -0.45
206116845 8.89 -4.92 6.10 7.07 6.91 7.24 0.09 -0.97 0.00 1.58 0.00 -0.62
206143369 7.22 -6.92 8.51 6.89 6.75 7.02 0.07 1.62 0.00 1.58 0.00 1.03
206453445 10.00 5.08 7.54 7.05 6.85 7.25 0.10 0.49 0.00 1.58 0.00 0.31
206477245 7.22 0.08 5.68 6.77 6.65 6.89 0.06 -1.09 0.00 1.58 0.00 -0.69
206777809 5.00 3.08 8.24 6.42 6.27 6.58 0.08 1.82 0.00 1.58 0.00 1.15
206794809 9.44 -16.92 7.31 7.35 7.13 7.57 0.11 -0.04 0.00 1.58 0.00 -0.03
206799565 6.67 -24.92 6.54 7.12 6.92 7.32 0.10 -0.58 0.00 1.58 0.00 -0.37
206842413 7.78 -28.92 8.65 7.34 7.11 7.56 0.11 1.32 0.01 1.58 0.00 0.83
207166085 8.89 -13.92 8.23 7.23 7.04 7.42 0.10 1.00 0.00 1.58 0.00 0.64
207238845 9.44 -2.92 8.77 7.11 6.93 7.30 0.09 1.66 0.00 1.58 0.00 1.05
207301409 7.78 -12.92 5.44 7.06 6.90 7.22 0.08 -1.62 0.00 1.58 0.00 -1.03
207448965 3.89 -4.92 6.69 6.41 6.20 6.62 0.11 0.28 0.00 1.58 0.00 0.18
207561849 6.11 -16.92 7.42 6.91 6.73 7.09 0.09 0.51 0.00 1.58 0.00 0.32
207793725 8.33 -17.92 6.08 7.22 7.03 7.41 0.10 -1.14 0.00 1.58 0.00 -0.72
217739413 7.78 -19.92 6.59 7.18 6.99 7.37 0.10 -0.59 0.00 1.58 0.00 -0.37
217799925 8.33 11.08 6.69 6.73 6.59 6.86 0.07 -0.03 0.00 1.58 0.00 -0.02
218184809 5.56 5.08 4.62 6.46 6.33 6.60 0.07 -1.85 0.00 1.58 0.00 -1.17
218398329 5.00 -9.92 2.88 6.64 6.46 6.82 0.09 -3.76 0.00 1.58 0.01 -2.38
218472449 7.22 -21.92 8.46 7.14 6.95 7.33 0.10 1.32 0.00 1.58 0.00 0.83
218496245 8.89 23.08 3.96 6.60 6.43 6.76 0.09 -2.64 0.00 1.58 0.00 -1.67
218575129 6.11 10.08 8.74 6.45 6.33 6.57 0.06 2.29 0.00 1.58 0.00 1.44
224406809 8.89 9.08 8.24 6.83 6.68 6.99 0.08 1.41 0.00 1.58 0.00 0.89
224431969 7.78 6.08 7.91 6.74 6.62 6.86 0.06 1.17 0.00 1.58 0.00 0.74
224438769 7.78 1.08 8.08 6.82 6.70 6.95 0.06 1.25 0.00 1.58 0.00 0.79
224629233 8.33 -21.92 8.77 7.29 7.08 7.49 0.10 1.48 0.00 1.58 0.00 0.94
224835205 8.33 34.08 5.44 6.34 6.15 6.52 0.09 -0.90 0.00 1.58 0.00 -0.57
225064365 5.56 6.08 5.38 6.44 6.31 6.58 0.07 -1.06 0.00 1.58 0.00 -0.67
231336009 8.89 -10.92 9.23 7.18 6.99 7.36 0.09 2.06 0.00 1.58 0.00 1.30
231421685 6.11 19.08 6.23 6.30 6.17 6.42 0.06 -0.07 0.00 1.58 0.00 -0.04
231577405 6.11 13.08 8.31 6.40 6.28 6.52 0.06 1.91 0.00 1.58 0.00 1.21
231665133 7.78 -1.92 8.08 6.88 6.74 7.01 0.07 1.20 0.00 1.58 0.00 0.76
232047973 7.78 -29.92 6.92 7.35 7.12 7.58 0.12 -0.43 0.01 1.58 0.00 -0.27
272639209 7.22 6.08 7.31 6.67 6.55 6.78 0.06 0.64 0.00 1.58 0.00 0.41
272854769 5.00 7.08 6.69 6.35 6.20 6.51 0.08 0.34 0.00 1.58 0.00 0.21
272871085 10.00 1.08 4.12 7.12 6.91 7.32 0.10 -3.00 0.00 1.58 0.01 -1.90
272934329 8.33 8.08 8.92 6.78 6.65 6.91 0.07 2.14 0.00 1.58 0.00 1.36
273243725 6.67 18.08 9.23 6.39 6.27 6.50 0.06 2.84 0.00 1.58 0.00 1.80
273463365 4.44 17.08 6.69 6.11 5.93 6.29 0.09 0.58 0.00 1.58 0.00 0.37
273469489 7.78 -2.92 7.62 6.89 6.76 7.02 0.07 0.72 0.00 1.58 0.00 0.46
273931209 8.33 16.08 5.69 6.64 6.51 6.78 0.07 -0.95 0.00 1.58 0.00 -0.60
274240609 6.67 -4.92 5.27 6.78 6.65 6.91 0.07 -1.50 0.00 1.58 0.00 -0.95
274416049 7.78 22.08 7.77 6.47 6.33 6.60 0.07 1.30 0.00 1.58 0.00 0.82
274648605 3.33 -14.92 7.69 6.51 6.25 6.76 0.13 1.18 0.01 1.58 0.00 0.75
274847169 7.22 -7.92 4.62 6.90 6.76 7.04 0.07 -2.29 0.00 1.58 0.00 -1.45
275010365 8.33 -10.92 5.46 7.10 6.94 7.27 0.09 -1.64 0.00 1.58 0.00 -1.04
275120529 8.89 -16.92 8.08 7.28 7.08 7.48 0.10 0.80 0.00 1.58 0.00 0.51
275210289 6.67 0.08 7.85 6.69 6.58 6.81 0.06 1.15 0.00 1.58 0.00 0.73
275336097 5.56 -23.92 5.54 6.96 6.74 7.17 0.11 -1.42 0.00 1.58 0.00 -0.90
275661805 8.33 -21.92 8.33 7.29 7.08 7.49 0.10 1.04 0.00 1.58 0.00 0.66
275812765 6.67 -20.92 7.27 7.05 6.87 7.24 0.10 0.22 0.00 1.58 0.00 0.14
275876685 5.56 -16.92 7.34 6.84 6.65 7.02 0.10 0.51 0.00 1.58 0.00 0.32
275930405 7.22 -8.92 7.69 6.92 6.78 7.06 0.07 0.77 0.00 1.58 0.00 0.49
285701325 8.89 27.08 7.25 6.53 6.35 6.70 0.09 0.72 0.00 1.58 0.00 0.46
285744853 10.00 -21.92 7.69 7.51 7.26 7.76 0.13 0.18 0.01 1.58 0.00 0.12
285844129 3.33 11.08 4.29 6.06 5.84 6.29 0.11 -1.78 0.01 1.58 0.00 -1.13
286191605 10.00 4.08 6.73 7.07 6.87 7.27 0.10 -0.34 0.00 1.58 0.00 -0.21
286303129 10.00 -1.92 6.26 7.17 6.96 7.38 0.11 -0.91 0.00 1.58 0.00 -0.57
286407169 6.67 -17.92 6.92 7.00 6.83 7.18 0.09 -0.08 0.00 1.58 0.00 -0.05
292975969 10.00 19.08 6.15 6.81 6.61 7.02 0.11 -0.66 0.00 1.58 0.00 -0.42
293086125 7.78 -6.92 7.58 6.96 6.82 7.10 0.07 0.62 0.00 1.58 0.00 0.39
293152765 6.11 20.08 7.31 6.28 6.15 6.41 0.07 1.03 0.00 1.58 0.00 0.65
299308129 10.00 5.08 4.85 7.05 6.85 7.25 0.10 -2.20 0.00 1.58 0.00 -1.39
299308141 6.11 -26.92 9.23 7.08 6.86 7.30 0.11 2.15 0.00 1.58 0.00 1.36
299310849 5.56 -5.92 7.15 6.65 6.50 6.80 0.08 0.50 0.00 1.58 0.00 0.32
299316965 10.00 20.08 8.77 6.79 6.59 7.00 0.11 1.97 0.00 1.58 0.00 1.25
299653569 7.22 -16.92 9.23 7.06 6.89 7.23 0.09 2.17 0.00 1.58 0.00 1.37
300162885 10.00 -6.92 7.38 7.25 7.04 7.47 0.11 0.13 0.00 1.58 0.00 0.08
300166965 8.33 -26.92 6.35 7.37 7.15 7.60 0.11 -1.03 0.01 1.58 0.00 -0.65
340124445 8.89 9.08 6.77 6.83 6.68 6.99 0.08 -0.07 0.00 1.58 0.00 -0.04
340214205 8.33 36.08 7.08 6.30 6.11 6.49 0.10 0.78 0.00 1.58 0.00 0.49
340397805 5.56 19.08 6.69 6.22 6.08 6.36 0.07 0.47 0.00 1.58 0.00 0.30
340480089 8.33 8.08 7.15 6.78 6.65 6.91 0.07 0.38 0.00 1.58 0.00 0.24
340505245 7.78 36.08 8.92 6.23 6.05 6.41 0.09 2.70 0.00 1.58 0.00 1.70
340603853 10.00 -7.92 6.92 7.27 7.05 7.49 0.11 -0.35 0.00 1.58 0.00 -0.22
340826205 7.78 11.08 6.51 6.65 6.54 6.77 0.06 -0.14 0.00 1.58 0.00 -0.09
341005049 8.89 5.08 6.62 6.90 6.75 7.06 0.08 -0.29 0.00 1.58 0.00 -0.18
341005725 7.22 16.08 6.08 6.49 6.38 6.61 0.06 -0.42 0.00 1.58 0.00 -0.26
341090045 7.78 16.08 7.92 6.57 6.45 6.69 0.06 1.35 0.00 1.58 0.00 0.86
341169605 8.33 -2.92 6.46 6.97 6.82 7.11 0.07 -0.50 0.00 1.58 0.00 -0.32
341575569 6.11 -7.92 7.87 6.76 6.61 6.90 0.07 1.11 0.00 1.58 0.00 0.70
341930529 5.56 -1.92 9.23 6.58 6.44 6.72 0.07 2.65 0.00 1.58 0.00 1.68
341946849 10.00 10.08 5.98 6.96 6.76 7.17 0.10 -0.99 0.00 1.58 0.00 -0.62
342417413 7.22 -12.92 6.06 6.99 6.83 7.15 0.08 -0.93 0.00 1.58 0.00 -0.59
342928769 6.67 -20.92 6.35 7.05 6.87 7.24 0.10 -0.71 0.00 1.58 0.00 -0.45
343063413 6.11 -27.92 7.69 7.10 6.88 7.32 0.11 0.59 0.01 1.58 0.00 0.38
343189889 6.11 -19.92 2.31 6.96 6.77 7.15 0.10 -4.65 0.00 1.58 0.01 -2.94
343292569 7.22 28.08 7.01 6.29 6.15 6.43 0.07 0.72 0.00 1.58 0.00 0.45
343323849 8.89 12.08 6.85 6.78 6.63 6.94 0.08 0.06 0.00 1.58 0.00 0.04
354084165 6.11 -3.92 7.12 6.69 6.55 6.82 0.07 0.43 0.00 1.58 0.00 0.27
354128369 8.89 22.08 5.93 6.61 6.45 6.78 0.08 -0.68 0.00 1.58 0.00 -0.43
361072529 7.78 -20.92 9.23 7.20 7.01 7.39 0.10 2.03 0.00 1.58 0.00 1.29
361248645 3.89 -6.92 7.75 6.45 6.23 6.66 0.11 1.30 0.00 1.58 0.00 0.82
361303045 3.33 -14.92 6.76 6.51 6.25 6.76 0.13 0.25 0.01 1.58 0.00 0.16
367552253 7.22 -29.92 0.00 7.28 7.05 7.50 0.11 -7.28 0.01 1.56 0.04 -4.61
367642009 9.44 -0.92 8.31 7.08 6.90 7.26 0.09 1.23 0.00 1.58 0.00 0.78
368104409 5.56 24.08 3.92 6.14 5.99 6.29 0.08 -2.21 0.00 1.58 0.00 -1.40
368322005 6.11 -11.92 8.31 6.82 6.67 6.98 0.08 1.48 0.00 1.58 0.00 0.94
408255005 6.67 10.08 7.00 6.52 6.41 6.63 0.06 0.48 0.00 1.58 0.00 0.30
408306693 6.67 -25.92 6.69 7.14 6.93 7.34 0.11 -0.44 0.00 1.58 0.00 -0.28
408420925 5.56 13.08 6.62 6.33 6.19 6.46 0.07 0.29 0.00 1.58 0.00 0.18
408565769 6.67 18.08 7.54 6.39 6.27 6.50 0.06 1.15 0.00 1.58 0.00 0.73
408811929 5.56 14.08 3.23 6.31 6.17 6.44 0.07 -3.08 0.00 1.58 0.00 -1.94
408869725 6.67 -1.92 7.46 6.73 6.61 6.85 0.06 0.73 0.00 1.58 0.00 0.46
408879245 10.00 16.08 5.93 6.86 6.66 7.07 0.10 -0.93 0.00 1.58 0.00 -0.59
408982605 8.89 -6.92 7.00 7.11 6.93 7.28 0.09 -0.11 0.00 1.58 0.00 -0.07
409253381 4.44 -29.92 6.92 6.91 6.65 7.17 0.13 0.01 0.01 1.58 0.00 0.01
409379045 4.44 11.08 7.15 6.21 6.04 6.39 0.09 0.94 0.00 1.58 0.00 0.60
409513689 6.67 -5.92 7.01 6.80 6.66 6.93 0.07 0.22 0.00 1.58 0.00 0.14
409594605 5.56 -4.92 8.31 6.63 6.48 6.78 0.08 1.68 0.00 1.58 0.00 1.06
409960449 7.78 -12.92 6.06 7.06 6.90 7.22 0.08 -1.00 0.00 1.58 0.00 -0.64
409963165 8.33 -9.92 7.23 7.09 6.92 7.25 0.08 0.15 0.00 1.58 0.00 0.09
410160433 6.67 -24.92 7.88 7.12 6.92 7.32 0.10 0.76 0.00 1.58 0.00 0.48
410214089 8.33 -1.92 9.08 6.95 6.80 7.09 0.07 2.13 0.00 1.58 0.00 1.35
410584689 7.22 -10.92 8.33 6.96 6.81 7.10 0.08 1.38 0.00 1.58 0.00 0.87
410636373 6.11 6.08 8.41 6.52 6.40 6.64 0.06 1.89 0.00 1.58 0.00 1.19
410830165 8.89 -8.92 7.92 7.14 6.96 7.32 0.09 0.78 0.00 1.58 0.00 0.49
411298005 5.00 -5.92 7.38 6.58 6.41 6.75 0.09 0.81 0.00 1.58 0.00 0.51
411345605 8.89 -16.92 6.76 7.28 7.08 7.48 0.10 -0.52 0.00 1.58 0.00 -0.33
411568649 8.33 -10.92 7.08 7.10 6.94 7.27 0.09 -0.03 0.00 1.58 0.00 -0.02
411578165 6.67 -11.92 6.62 6.90 6.75 7.05 0.08 -0.28 0.00 1.58 0.00 -0.18
411693085 4.44 -7.92 6.54 6.54 6.34 6.73 0.10 0.00 0.00 1.58 0.00 0.00
421640805 10.00 32.08 6.15 6.59 6.36 6.82 0.12 -0.44 0.01 1.58 0.00 -0.28
421726485 5.56 13.08 3.46 6.33 6.19 6.46 0.07 -2.86 0.00 1.58 0.00 -1.81
422010049 5.00 -9.92 6.92 6.64 6.46 6.82 0.09 0.28 0.00 1.58 0.00 0.18
422067173 5.00 -26.92 5.38 6.93 6.70 7.17 0.12 -1.55 0.01 1.58 0.00 -0.98
422150129 8.33 -1.92 7.25 6.95 6.80 7.09 0.07 0.30 0.00 1.58 0.00 0.19
422325565 8.33 -2.92 8.24 6.97 6.82 7.11 0.07 1.28 0.00 1.58 0.00 0.81
422409885 8.89 17.08 8.57 6.70 6.54 6.86 0.08 1.87 0.00 1.58 0.00 1.18
422447289 3.89 29.08 9.23 5.83 5.61 6.05 0.11 3.40 0.01 1.58 0.01 2.15
422571045 6.11 -26.92 7.79 7.08 6.86 7.30 0.11 0.71 0.00 1.58 0.00 0.45
428476165 8.89 23.08 6.92 6.60 6.43 6.76 0.09 0.33 0.00 1.58 0.00 0.21
428536693 6.67 -25.92 6.23 7.14 6.93 7.34 0.11 -0.91 0.00 1.58 0.00 -0.57
428621009 5.56 18.08 4.45 6.24 6.10 6.38 0.07 -1.79 0.00 1.58 0.00 -1.13
428649577 6.67 -29.92 9.23 7.21 6.98 7.43 0.11 2.03 0.01 1.58 0.00 1.28
428820925 10.00 -7.92 5.77 7.27 7.05 7.49 0.11 -1.50 0.00 1.58 0.00 -0.95
428923609 6.11 21.08 5.27 6.26 6.13 6.39 0.07 -0.99 0.00 1.58 0.00 -0.62
429228929 5.56 -5.92 9.23 6.65 6.50 6.80 0.08 2.58 0.00 1.58 0.00 1.63
435301325 8.33 24.08 6.23 6.51 6.35 6.66 0.08 -0.27 0.00 1.58 0.00 -0.17
435357089 6.67 26.08 6.23 6.25 6.11 6.39 0.07 -0.02 0.00 1.58 0.00 -0.01
435463845 8.89 19.08 8.31 6.66 6.50 6.82 0.08 1.64 0.00 1.58 0.00 1.04
435748085 5.00 -1.92 7.15 6.51 6.35 6.67 0.08 0.65 0.00 1.58 0.00 0.41
435936445 7.78 -10.92 9.00 7.03 6.87 7.18 0.08 1.97 0.00 1.58 0.00 1.25
435955485 10.00 -4.92 4.85 7.22 7.01 7.43 0.11 -2.37 0.00 1.58 0.00 -1.50
435960925 7.78 -18.92 8.77 7.16 6.98 7.35 0.09 1.60 0.00 1.58 0.00 1.01
436054085 5.56 -5.92 8.85 6.65 6.50 6.80 0.08 2.20 0.00 1.58 0.00 1.39
476032645 8.89 8.08 6.00 6.85 6.70 7.00 0.08 -0.85 0.00 1.58 0.00 -0.54
476289689 8.89 4.08 8.62 6.92 6.77 7.07 0.08 1.70 0.00 1.58 0.00 1.07
476371285 5.56 0.08 7.31 6.55 6.41 6.69 0.07 0.76 0.00 1.58 0.00 0.48
476651445 9.44 -3.92 5.85 7.13 6.94 7.32 0.10 -1.28 0.00 1.58 0.00 -0.81
476950645 10.00 18.08 7.54 6.83 6.62 7.03 0.10 0.71 0.00 1.58 0.00 0.45
477015925 9.44 26.08 6.54 6.62 6.42 6.81 0.10 -0.08 0.00 1.58 0.00 -0.05
477105685 5.56 28.08 7.08 6.07 5.91 6.23 0.08 1.01 0.00 1.58 0.00 0.64
477174369 8.33 -5.92 7.31 7.02 6.86 7.17 0.08 0.29 0.00 1.58 0.00 0.18
477285885 5.56 -12.92 8.15 6.77 6.60 6.94 0.09 1.39 0.00 1.58 0.00 0.88
477285889 5.00 18.08 4.79 6.17 6.01 6.32 0.08 -1.37 0.00 1.58 0.00 -0.87
477644929 6.67 0.08 7.87 6.69 6.58 6.81 0.06 1.17 0.00 1.58 0.00 0.74
477752369 7.22 6.08 8.77 6.67 6.55 6.78 0.06 2.10 0.00 1.58 0.00 1.33
477955685 9.44 10.08 7.09 6.89 6.72 7.07 0.09 0.20 0.00 1.58 0.00 0.12
478365045 7.22 12.08 6.00 6.56 6.45 6.67 0.06 -0.56 0.00 1.58 0.00 -0.36
478415365 6.67 -16.92 7.09 6.98 6.81 7.15 0.09 0.10 0.00 1.58 0.00 0.07
478462965 8.89 -7.92 8.92 7.12 6.95 7.30 0.09 1.80 0.00 1.58 0.00 1.14
478486093 5.56 -25.92 7.85 6.99 6.77 7.21 0.11 0.86 0.01 1.58 0.00 0.54
478581965 7.22 -16.92 5.19 7.06 6.89 7.23 0.09 -1.86 0.00 1.58 0.00 -1.18
478631605 7.78 -16.92 6.23 7.13 6.95 7.31 0.09 -0.90 0.00 1.58 0.00 -0.57
478735645 5.00 34.08 5.54 5.89 5.70 6.09 0.10 -0.36 0.00 1.58 0.00 -0.22
478759445 7.78 0.08 3.23 6.84 6.71 6.97 0.06 -3.61 0.00 1.58 0.00 -2.28
478868929 8.89 -19.92 6.23 7.33 7.12 7.54 0.11 -1.10 0.00 1.58 0.00 -0.70
478913805 3.89 -13.92 5.77 6.56 6.33 6.80 0.12 -0.80 0.01 1.58 0.00 -0.50
479181725 5.56 5.08 7.38 6.46 6.33 6.60 0.07 0.92 0.00 1.58 0.00 0.58
479771285 6.11 -24.92 7.31 7.05 6.84 7.25 0.11 0.26 0.00 1.58 0.00 0.17
479995009 3.33 -22.92 0.00 6.64 6.37 6.92 0.14 -6.64 0.01 1.57 0.05 -4.21
480035129 6.67 -15.92 8.08 6.97 6.80 7.13 0.09 1.11 0.00 1.58 0.00 0.70
489801289 5.00 12.08 2.97 6.27 6.12 6.42 0.08 -3.30 0.00 1.58 0.00 -2.09
489933205 7.22 10.08 5.60 6.60 6.49 6.71 0.06 -0.99 0.00 1.58 0.00 -0.63
490441169 6.11 -5.92 7.69 6.72 6.58 6.86 0.07 0.97 0.00 1.58 0.00 0.61
490509845 5.56 -13.92 6.69 6.79 6.61 6.96 0.09 -0.09 0.00 1.58 0.00 -0.06
496851525 8.33 1.08 8.90 6.90 6.76 7.04 0.07 2.00 0.00 1.58 0.00 1.27
497127605 6.11 6.08 9.23 6.52 6.40 6.64 0.06 2.71 0.00 1.58 0.00 1.71
497219405 7.78 -20.92 8.24 7.20 7.01 7.39 0.10 1.04 0.00 1.58 0.00 0.66
497360169 7.22 -6.92 7.58 6.89 6.75 7.02 0.07 0.70 0.00 1.58 0.00 0.44
503222453 5.00 -23.92 7.50 6.88 6.66 7.11 0.11 0.62 0.01 1.58 0.00 0.39
503455689 3.89 -17.92 7.38 6.63 6.39 6.87 0.12 0.75 0.01 1.58 0.00 0.48
503630445 10.00 28.08 8.46 6.66 6.44 6.88 0.11 1.80 0.01 1.58 0.00 1.14
503641329 8.33 -3.92 9.00 6.98 6.83 7.13 0.08 2.02 0.00 1.58 0.00 1.28
503901765 10.00 -16.92 8.08 7.43 7.19 7.66 0.12 0.65 0.01 1.58 0.00 0.41
503901769 10.00 -17.92 6.00 7.44 7.20 7.68 0.12 -1.44 0.01 1.58 0.00 -0.91
504168329 6.67 5.08 8.54 6.61 6.50 6.72 0.06 1.93 0.00 1.58 0.00 1.22
544312125 4.44 35.08 5.08 5.80 5.59 6.02 0.11 -0.73 0.00 1.58 0.00 -0.46
544592289 6.67 4.08 7.23 6.63 6.51 6.74 0.06 0.60 0.00 1.58 0.00 0.38
544612005 5.56 18.08 7.77 6.24 6.10 6.38 0.07 1.53 0.00 1.58 0.00 0.97
544836405 5.56 5.08 8.85 6.46 6.33 6.60 0.07 2.38 0.00 1.58 0.00 1.51
544839809 6.11 34.08 6.31 6.04 5.87 6.21 0.09 0.27 0.00 1.58 0.00 0.17
544840485 9.44 27.08 7.69 6.60 6.40 6.80 0.10 1.09 0.00 1.58 0.00 0.69
544841165 6.11 8.08 5.92 6.48 6.37 6.60 0.06 -0.56 0.00 1.58 0.00 -0.35
544906449 7.22 3.08 7.08 6.72 6.60 6.83 0.06 0.36 0.00 1.58 0.00 0.23
545083925 8.33 10.08 8.92 6.74 6.61 6.88 0.07 2.18 0.00 1.58 0.00 1.38
545145809 6.11 23.08 5.62 6.23 6.09 6.36 0.07 -0.61 0.00 1.58 0.00 -0.39
545326685 7.22 -2.92 6.54 6.82 6.69 6.94 0.06 -0.28 0.00 1.58 0.00 -0.18
545428689 6.67 -1.92 7.50 6.73 6.61 6.85 0.06 0.77 0.00 1.58 0.00 0.49
545477649 6.11 -3.92 7.31 6.69 6.55 6.82 0.07 0.62 0.00 1.58 0.00 0.39
545502125 8.33 5.08 7.85 6.83 6.70 6.96 0.07 1.02 0.00 1.58 0.00 0.64
545675525 7.78 9.08 5.85 6.69 6.57 6.81 0.06 -0.84 0.00 1.58 0.00 -0.53
545691165 6.11 -14.92 7.46 6.88 6.71 7.04 0.09 0.59 0.00 1.58 0.00 0.37
545765969 7.78 -0.92 7.54 6.86 6.73 6.99 0.07 0.69 0.00 1.58 0.00 0.43
545919645 8.89 -10.92 7.08 7.18 6.99 7.36 0.09 -0.10 0.00 1.58 0.00 -0.06
546120249 6.11 23.08 5.77 6.23 6.09 6.36 0.07 -0.46 0.00 1.58 0.00 -0.29
547485685 8.89 -22.92 8.94 7.38 7.16 7.60 0.11 1.56 0.01 1.58 0.00 0.99
547650925 5.00 -20.92 7.62 6.83 6.62 7.05 0.11 0.78 0.00 1.58 0.00 0.50
547688325 6.11 -20.92 6.92 6.98 6.79 7.17 0.10 -0.06 0.00 1.58 0.00 -0.03
547737965 7.78 -19.92 7.76 7.18 6.99 7.37 0.10 0.58 0.00 1.58 0.00 0.37
547784885 8.89 -25.92 8.54 7.43 7.20 7.66 0.12 1.11 0.01 1.58 0.00 0.70
548026285 7.22 -19.92 7.87 7.11 6.93 7.29 0.09 0.76 0.00 1.58 0.00 0.48
557614285 5.00 24.08 4.12 6.06 5.90 6.23 0.09 -1.94 0.00 1.58 0.00 -1.23
558048125 10.00 12.08 8.90 6.93 6.73 7.13 0.10 1.97 0.00 1.58 0.00 1.25
558165777 6.67 -23.92 8.77 7.10 6.90 7.30 0.10 1.67 0.00 1.58 0.00 1.05
558180045 10.00 23.08 8.85 6.74 6.53 6.96 0.11 2.10 0.00 1.58 0.00 1.33
564670653 5.00 -28.92 9.23 6.97 6.72 7.21 0.12 2.26 0.01 1.58 0.00 1.43
564904633 3.33 -24.92 7.31 6.68 6.39 6.96 0.14 0.63 0.01 1.58 0.00 0.40
564932445 8.33 -2.92 6.76 6.97 6.82 7.11 0.07 -0.21 0.00 1.58 0.00 -0.13
571289769 6.11 -8.92 5.54 6.77 6.63 6.92 0.08 -1.24 0.00 1.58 0.00 -0.78
571535925 8.89 -8.92 8.54 7.14 6.96 7.32 0.09 1.40 0.00 1.58 0.00 0.88
612359049 7.78 15.08 7.15 6.59 6.46 6.71 0.06 0.57 0.00 1.58 0.00 0.36
612506609 6.11 -1.92 6.54 6.65 6.52 6.78 0.07 -0.12 0.00 1.58 0.00 -0.07
612677965 6.11 -1.92 7.23 6.65 6.52 6.78 0.07 0.58 0.00 1.58 0.00 0.36
612688173 5.56 -24.92 7.38 6.97 6.76 7.19 0.11 0.41 0.00 1.58 0.00 0.26
612902369 8.33 19.08 7.85 6.59 6.45 6.73 0.07 1.26 0.00 1.58 0.00 0.79
613008445 8.33 -5.92 6.23 7.02 6.86 7.17 0.08 -0.79 0.00 1.58 0.00 -0.50
613228085 8.89 5.08 7.17 6.90 6.75 7.06 0.08 0.27 0.00 1.58 0.00 0.17
613271609 7.78 23.08 7.85 6.45 6.31 6.59 0.07 1.40 0.00 1.58 0.00 0.88
613466085 6.67 -5.92 7.38 6.80 6.66 6.93 0.07 0.59 0.00 1.58 0.00 0.37
613581685 9.44 7.08 8.34 6.94 6.77 7.12 0.09 1.40 0.00 1.58 0.00 0.89
613692525 3.89 3.08 6.69 6.27 6.07 6.48 0.10 0.42 0.00 1.58 0.00 0.26
614238569 5.00 -16.92 7.58 6.76 6.56 6.96 0.10 0.82 0.00 1.58 0.00 0.52
614862125 7.22 27.08 5.46 6.31 6.17 6.45 0.07 -0.85 0.00 1.58 0.00 -0.53
615039609 8.33 -17.92 9.23 7.22 7.03 7.41 0.10 2.01 0.00 1.58 0.00 1.27
615198725 7.78 -13.92 6.54 7.08 6.91 7.25 0.08 -0.54 0.00 1.58 0.00 -0.34
615199405 5.00 -24.92 7.12 6.90 6.67 7.13 0.12 0.22 0.01 1.58 0.00 0.14
615399325 8.33 -1.92 8.00 6.95 6.80 7.09 0.07 1.05 0.00 1.58 0.00 0.66
615994325 8.89 -6.92 7.69 7.11 6.93 7.28 0.09 0.58 0.00 1.58 0.00 0.37
625627213 5.00 -29.92 0.00 6.98 6.74 7.23 0.13 -6.98 0.01 1.57 0.04 -4.42
625648293 2.78 -28.92 7.50 6.67 6.36 6.99 0.16 0.83 0.01 1.58 0.00 0.52
625662565 8.89 15.08 6.92 6.73 6.58 6.89 0.08 0.19 0.00 1.58 0.00 0.12
626203845 9.44 -14.92 7.69 7.32 7.10 7.53 0.11 0.37 0.00 1.58 0.00 0.24
632610805 6.11 -0.92 6.10 6.64 6.51 6.77 0.07 -0.54 0.00 1.58 0.00 -0.34
632627125 8.33 32.08 7.75 6.37 6.19 6.54 0.09 1.38 0.00 1.58 0.00 0.87
632680845 5.00 -13.92 7.91 6.71 6.52 6.90 0.10 1.20 0.00 1.58 0.00 0.76
632699885 8.33 -2.92 6.59 6.97 6.82 7.11 0.07 -0.37 0.00 1.58 0.00 -0.24
632754289 3.89 14.08 9.73 6.09 5.89 6.29 0.10 3.64 0.00 1.58 0.01 2.30
633224845 8.89 -7.92 8.57 7.12 6.95 7.30 0.09 1.45 0.00 1.58 0.00 0.92
639259169 6.11 31.08 6.92 6.09 5.93 6.25 0.08 0.83 0.00 1.58 0.00 0.53
639268005 10.00 37.08 7.62 6.50 6.26 6.75 0.12 1.11 0.01 1.58 0.00 0.70
639290445 6.11 16.08 7.38 6.35 6.23 6.47 0.06 1.04 0.00 1.58 0.00 0.66
639559053 6.11 -17.92 7.31 6.93 6.75 7.11 0.09 0.38 0.00 1.58 0.00 0.24
639598489 6.11 -21.92 6.15 7.00 6.80 7.19 0.10 -0.84 0.00 1.58 0.00 -0.53
640069113 5.56 -27.92 4.62 7.02 6.80 7.25 0.12 -2.41 0.01 1.58 0.00 -1.53
680105405 5.00 35.08 5.46 5.88 5.68 6.07 0.10 -0.42 0.00 1.58 0.00 -0.26
680106773 8.89 -7.92 7.38 7.12 6.95 7.30 0.09 0.26 0.00 1.58 0.00 0.16
680525645 6.11 19.08 5.00 6.30 6.17 6.42 0.06 -1.30 0.00 1.58 0.00 -0.82
680670485 7.78 -6.92 9.08 6.96 6.82 7.10 0.07 2.12 0.00 1.58 0.00 1.34
681108417 6.11 -29.92 4.62 7.13 6.90 7.36 0.12 -2.52 0.01 1.58 0.00 -1.59
681141729 10.00 3.08 4.23 7.08 6.88 7.29 0.10 -2.85 0.00 1.58 0.00 -1.81
681232169 10.00 -0.92 7.09 7.15 6.95 7.36 0.11 -0.06 0.00 1.58 0.00 -0.04
681627245 7.22 1.08 7.54 6.75 6.63 6.87 0.06 0.79 0.00 1.58 0.00 0.50
681634045 4.44 -3.92 4.86 6.47 6.28 6.65 0.10 -1.61 0.00 1.58 0.00 -1.02
681636765 7.22 17.08 8.69 6.48 6.36 6.59 0.06 2.21 0.00 1.58 0.00 1.40
681636769 8.89 15.08 7.08 6.73 6.58 6.89 0.08 0.34 0.00 1.58 0.00 0.22
681703405 9.44 -0.92 7.01 7.08 6.90 7.26 0.09 -0.07 0.00 1.58 0.00 -0.05
681979489 7.22 -16.92 6.83 7.06 6.89 7.23 0.09 -0.22 0.00 1.58 0.00 -0.14
682074005 6.67 41.08 7.00 6.00 5.80 6.19 0.10 1.00 0.00 1.58 0.00 0.64
682232449 6.67 -7.92 7.99 6.83 6.69 6.97 0.07 1.16 0.00 1.58 0.00 0.73
682237893 6.67 -23.92 9.23 7.10 6.90 7.30 0.10 2.13 0.00 1.58 0.00 1.35
682490857 2.78 -28.92 7.50 6.67 6.36 6.99 0.16 0.83 0.01 1.58 0.00 0.52
682516689 6.11 7.08 6.25 6.50 6.38 6.62 0.06 -0.25 0.00 1.58 0.00 -0.16
682853289 8.33 -12.92 6.23 7.14 6.96 7.31 0.09 -0.91 0.00 1.58 0.00 -0.57
683302085 7.78 -16.92 6.51 7.13 6.95 7.31 0.09 -0.62 0.00 1.58 0.00 -0.39
683497933 6.67 -2.92 5.05 6.75 6.62 6.87 0.06 -1.70 0.00 1.58 0.00 -1.07
683650245 5.56 -15.92 7.01 6.82 6.64 7.00 0.09 0.19 0.00 1.58 0.00 0.12
683680165 7.22 -18.92 5.38 7.09 6.91 7.27 0.09 -1.71 0.00 1.58 0.00 -1.08
683986845 8.89 -16.92 6.35 7.28 7.08 7.48 0.10 -0.93 0.00 1.58 0.00 -0.59
693727169 5.00 14.08 7.42 6.23 6.08 6.39 0.08 1.18 0.00 1.58 0.00 0.75
693825765 3.33 18.08 8.41 5.95 5.72 6.17 0.12 2.46 0.01 1.58 0.00 1.56
693950209 7.78 30.08 7.58 6.33 6.17 6.49 0.08 1.25 0.00 1.58 0.00 0.79
693977409 6.11 -25.92 6.92 7.06 6.85 7.28 0.11 -0.14 0.00 1.58 0.00 -0.09
693991685 5.00 26.08 6.26 6.03 5.86 6.20 0.09 0.23 0.00 1.58 0.00 0.15
694179433 6.11 -27.92 9.23 7.10 6.88 7.32 0.11 2.13 0.01 1.58 0.00 1.35
694509165 6.11 -23.92 7.42 7.03 6.83 7.23 0.10 0.39 0.00 1.58 0.00 0.25
700828405 10.00 14.08 7.25 6.90 6.70 7.10 0.10 0.36 0.00 1.58 0.00 0.23
700976645 6.67 11.08 8.46 6.51 6.40 6.62 0.06 1.96 0.00 1.58 0.00 1.24
701021525 9.44 -20.92 6.26 7.42 7.19 7.65 0.12 -1.16 0.01 1.58 0.00 -0.73
701079329 5.00 -3.92 4.42 6.54 6.38 6.71 0.08 -2.12 0.00 1.58 0.00 -1.34
701442445 7.22 -27.92 8.08 7.24 7.03 7.46 0.11 0.83 0.00 1.58 0.00 0.53
707251693 1.11 -28.92 6.92 6.45 6.06 6.84 0.20 0.47 0.02 1.58 0.00 0.30
707506685 6.67 33.08 6.00 6.13 5.97 6.29 0.08 -0.13 0.00 1.58 0.00 -0.08
707574009 6.11 -13.92 6.63 6.86 6.69 7.02 0.08 -0.22 0.00 1.58 0.00 -0.14
707719525 3.33 4.08 2.77 6.18 5.96 6.41 0.12 -3.42 0.01 1.58 0.01 -2.16
707824245 5.00 12.08 7.15 6.27 6.12 6.42 0.08 0.89 0.00 1.58 0.00 0.56
708059525 4.44 15.08 4.15 6.14 5.97 6.32 0.09 -1.99 0.00 1.58 0.00 -1.26
708067689 10.00 -4.92 7.85 7.22 7.01 7.43 0.11 0.63 0.00 1.58 0.00 0.40
708249249 8.33 -1.92 6.00 6.95 6.80 7.09 0.07 -0.95 0.00 1.58 0.00 -0.60
748140765 6.11 25.08 6.15 6.19 6.05 6.33 0.07 -0.04 0.00 1.58 0.00 -0.03
748280849 7.22 1.08 5.08 6.75 6.63 6.87 0.06 -1.67 0.00 1.58 0.00 -1.06
748280853 5.56 -21.92 6.67 6.92 6.72 7.13 0.10 -0.26 0.00 1.58 0.00 -0.16
748617453 6.11 -28.92 8.65 7.11 6.89 7.34 0.11 1.54 0.01 1.58 0.00 0.97
748928205 5.56 7.08 6.54 6.43 6.30 6.56 0.07 0.11 0.00 1.58 0.00 0.07
748929565 8.33 11.08 2.69 6.73 6.59 6.86 0.07 -4.03 0.00 1.58 0.00 -2.55
749039045 3.89 6.08 6.00 6.22 6.02 6.42 0.10 -0.22 0.00 1.58 0.00 -0.14
749039049 6.11 -1.92 5.92 6.65 6.52 6.78 0.07 -0.73 0.00 1.58 0.00 -0.46
749152609 10.00 -6.92 8.69 7.25 7.04 7.47 0.11 1.44 0.00 1.58 0.00 0.91
749231489 8.33 30.08 8.77 6.40 6.23 6.57 0.09 2.37 0.00 1.58 0.00 1.50
749462685 6.67 26.08 5.23 6.25 6.11 6.39 0.07 -1.02 0.00 1.58 0.00 -0.64
749561289 5.00 -9.92 7.15 6.64 6.46 6.82 0.09 0.51 0.00 1.58 0.00 0.32
749657849 6.67 5.08 7.31 6.61 6.50 6.72 0.06 0.70 0.00 1.58 0.00 0.44
749782965 6.67 -18.92 5.67 7.02 6.84 7.20 0.09 -1.34 0.00 1.58 0.00 -0.85
749978805 8.33 12.08 5.62 6.71 6.58 6.84 0.07 -1.09 0.00 1.58 0.00 -0.69
749992409 9.44 14.08 4.46 6.82 6.65 7.00 0.09 -2.36 0.00 1.58 0.00 -1.49
750438485 5.00 26.08 7.62 6.03 5.86 6.20 0.09 1.59 0.00 1.58 0.00 1.00
750586725 4.44 -12.92 7.31 6.62 6.41 6.83 0.11 0.69 0.00 1.58 0.00 0.43
750645209 5.56 -19.92 8.24 6.89 6.69 7.09 0.10 1.35 0.00 1.58 0.00 0.86
750654733 7.78 -8.92 6.92 6.99 6.85 7.14 0.08 -0.07 0.00 1.58 0.00 -0.05
751584965 8.33 -13.92 8.46 7.15 6.98 7.33 0.09 1.31 0.00 1.58 0.00 0.83
761861125 7.22 0.08 7.25 6.77 6.65 6.89 0.06 0.49 0.00 1.58 0.00 0.31
762080765 8.89 -7.92 7.91 7.12 6.95 7.30 0.09 0.79 0.00 1.58 0.00 0.50
768580205 8.33 -23.92 8.65 7.32 7.11 7.54 0.11 1.33 0.00 1.58 0.00 0.84
768869209 9.44 -4.92 9.69 7.15 6.96 7.34 0.10 2.55 0.00 1.58 0.00 1.61
768875333 8.33 -26.92 9.23 7.37 7.15 7.60 0.11 1.86 0.01 1.58 0.00 1.17
768897765 5.56 25.08 7.09 6.12 5.97 6.27 0.08 0.97 0.00 1.58 0.00 0.61
769097005 7.22 -13.92 6.76 7.01 6.85 7.17 0.08 -0.25 0.00 1.58 0.00 -0.16
775535245 10.00 -7.92 5.77 7.27 7.05 7.49 0.11 -1.50 0.00 1.58 0.00 -0.95
775850085 2.78 15.08 4.85 5.92 5.67 6.18 0.13 -1.08 0.01 1.58 0.00 -0.68
776009889 7.22 -18.92 7.79 7.09 6.91 7.27 0.09 0.70 0.00 1.58 0.00 0.44
776311125 8.33 14.08 7.85 6.68 6.54 6.81 0.07 1.17 0.00 1.58 0.00 0.74
816076165 3.33 19.08 7.77 5.93 5.70 6.16 0.12 1.84 0.01 1.58 0.00 1.17
816192449 8.33 18.08 8.00 6.61 6.47 6.75 0.07 1.39 0.00 1.58 0.00 0.88
816663005 8.89 -10.92 8.23 7.18 6.99 7.36 0.09 1.06 0.00 1.58 0.00 0.67
816665045 8.89 4.08 8.15 6.92 6.77 7.07 0.08 1.23 0.00 1.58 0.00 0.78
816708565 4.44 14.08 7.08 6.16 5.99 6.34 0.09 0.92 0.00 1.58 0.00 0.58
817011165 7.78 8.08 7.00 6.70 6.59 6.82 0.06 0.30 0.00 1.58 0.00 0.19
817191365 6.67 -1.92 7.62 6.73 6.61 6.85 0.06 0.89 0.00 1.58 0.00 0.56
817226045 8.33 8.08 7.69 6.78 6.65 6.91 0.07 0.91 0.00 1.58 0.00 0.58
817515729 8.89 -5.92 8.08 7.09 6.92 7.26 0.09 0.99 0.00 1.58 0.00 0.62
817972689 8.89 -3.92 6.63 7.06 6.89 7.22 0.08 -0.42 0.00 1.58 0.00 -0.27
818157645 6.11 -20.92 7.27 6.98 6.79 7.17 0.10 0.29 0.00 1.58 0.00 0.18
818244005 5.56 -12.92 7.18 6.77 6.60 6.94 0.09 0.41 0.00 1.58 0.00 0.26
818424885 9.44 22.08 7.85 6.69 6.50 6.87 0.10 1.16 0.00 1.58 0.00 0.73
818918565 8.89 -2.92 6.68 7.04 6.88 7.20 0.08 -0.36 0.00 1.58 0.00 -0.23
819300733 7.78 -5.92 7.05 6.94 6.80 7.08 0.07 0.11 0.00 1.58 0.00 0.07
819410885 7.78 -17.92 6.92 7.15 6.97 7.33 0.09 -0.22 0.00 1.58 0.00 -0.14
819478885 10.00 -10.92 4.92 7.32 7.10 7.55 0.11 -2.40 0.01 1.58 0.00 -1.52
829604097 3.33 -27.92 6.54 6.73 6.44 7.02 0.15 -0.19 0.01 1.58 0.00 -0.12
830503049 8.89 34.08 5.27 6.41 6.21 6.61 0.10 -1.13 0.00 1.58 0.00 -0.72
830505765 7.78 -5.92 9.23 6.94 6.80 7.08 0.07 2.29 0.00 1.58 0.00 1.45
836702605 8.33 8.08 8.08 6.78 6.65 6.91 0.07 1.30 0.00 1.58 0.00 0.82
837375809 7.78 -6.92 6.59 6.96 6.82 7.10 0.07 -0.37 0.00 1.58 0.00 -0.23
843205445 6.67 -4.92 2.54 6.78 6.65 6.91 0.07 -4.24 0.00 1.58 0.00 -2.68
843260529 4.44 22.08 5.08 6.02 5.84 6.21 0.09 -0.95 0.00 1.58 0.00 -0.60
843677377 6.67 -29.92 6.92 7.21 6.98 7.43 0.11 -0.28 0.01 1.58 0.00 -0.18
844039805 10.00 29.08 6.00 6.64 6.42 6.86 0.11 -0.64 0.01 1.58 0.00 -0.41
59167 5.00 31.08 4.77 5.94 5.76 6.13 0.09 -1.18 0.00 1.58 0.00 -0.74
60527 9.44 0.08 7.92 7.06 6.88 7.24 0.09 0.86 0.00 1.58 0.00 0.54
82291 6.11 28.08 8.23 6.14 5.99 6.29 0.08 2.09 0.00 1.58 0.00 1.32
83647 5.56 26.08 9.00 6.10 5.95 6.26 0.08 2.90 0.00 1.58 0.00 1.83
276767 5.56 19.08 5.08 6.22 6.08 6.36 0.07 -1.15 0.00 1.58 0.00 -0.72
287651 0.56 12.08 5.36 5.68 5.31 6.05 0.19 -0.32 0.01 1.58 0.00 -0.21
310099 9.44 49.08 7.23 6.23 5.96 6.49 0.14 1.00 0.01 1.58 0.00 0.64
359051 10.00 29.08 3.38 6.64 6.42 6.86 0.11 -3.26 0.01 1.58 0.01 -2.06
421644 8.89 19.08 7.62 6.66 6.50 6.82 0.08 0.95 0.00 1.58 0.00 0.60
427731 4.44 43.08 4.46 5.67 5.43 5.91 0.12 -1.21 0.01 1.58 0.00 -0.76
480767 8.33 31.08 6.31 6.39 6.21 6.56 0.09 -0.08 0.00 1.58 0.00 -0.05
592967 10.00 31.08 7.23 6.61 6.38 6.84 0.12 0.62 0.01 1.58 0.00 0.39
593651 10.00 30.08 6.38 6.62 6.40 6.85 0.12 -0.24 0.01 1.58 0.00 -0.15
605891 10.00 32.08 8.15 6.59 6.36 6.82 0.12 1.56 0.01 1.58 0.00 0.99
619491 9.44 38.08 8.00 6.41 6.19 6.64 0.12 1.59 0.01 1.58 0.00 1.00
654167 7.22 53.08 6.15 5.86 5.62 6.11 0.12 0.29 0.01 1.58 0.00 0.18
654847 3.89 31.08 7.31 5.80 5.57 6.02 0.12 1.51 0.01 1.58 0.00 0.96
667087 5.00 30.08 7.54 5.96 5.78 6.14 0.09 1.58 0.00 1.58 0.00 1.00
686131 8.89 39.08 5.38 6.32 6.11 6.54 0.11 -0.94 0.00 1.58 0.00 -0.59
725571 3.33 42.08 6.18 5.54 5.26 5.81 0.14 0.64 0.01 1.58 0.00 0.41
793567 10.00 31.08 6.15 6.61 6.38 6.84 0.12 -0.45 0.01 1.58 0.00 -0.29
941807 4.44 30.08 5.08 5.89 5.69 6.09 0.10 -0.81 0.00 1.58 0.00 -0.51
955407 6.67 23.08 5.54 6.30 6.17 6.43 0.07 -0.76 0.00 1.58 0.00 -0.48
1073047 6.67 29.08 5.85 6.20 6.05 6.35 0.07 -0.35 0.00 1.58 0.00 -0.22
1079171 5.00 23.08 3.13 6.08 5.92 6.25 0.08 -2.95 0.00 1.58 0.00 -1.86
1120651 7.22 37.08 6.00 6.14 5.96 6.31 0.09 -0.14 0.00 1.58 0.00 -0.09
1146487 4.17 39.08 6.92 5.70 5.46 5.94 0.12 1.22 0.01 1.58 0.00 0.78
1160767 6.67 17.08 6.15 6.40 6.29 6.52 0.06 -0.25 0.00 1.58 0.00 -0.16
1168927 7.78 6.08 4.62 6.74 6.62 6.86 0.06 -2.12 0.00 1.58 0.00 -1.34
1203611 3.33 27.08 5.69 5.79 5.55 6.03 0.12 -0.10 0.01 1.58 0.00 -0.06
1227407 2.78 30.08 5.54 5.67 5.40 5.94 0.14 -0.13 0.01 1.58 0.00 -0.08
1234887 10.00 31.08 8.38 6.61 6.38 6.84 0.12 1.78 0.01 1.58 0.00 1.13
1236247 3.89 46.08 7.85 5.54 5.27 5.81 0.14 2.30 0.01 1.58 0.01 1.46
1236931 7.78 43.08 6.77 6.11 5.90 6.31 0.10 0.66 0.00 1.58 0.00 0.42
1258084 8.89 27.08 4.85 6.53 6.35 6.70 0.09 -1.68 0.00 1.58 0.00 -1.06
1283167 6.67 32.08 6.69 6.15 5.99 6.31 0.08 0.54 0.00 1.58 0.00 0.34
1483087 6.67 36.08 3.00 6.08 5.91 6.25 0.09 -3.08 0.00 1.58 0.00 -1.95
1498727 6.11 34.08 6.23 6.04 5.87 6.21 0.09 0.19 0.00 1.58 0.00 0.12
1532047 5.56 0.08 4.00 6.55 6.41 6.69 0.07 -2.55 0.00 1.58 0.00 -1.61
1636087 5.00 24.08 6.77 6.06 5.90 6.23 0.09 0.71 0.00 1.58 0.00 0.45
1640847 5.56 50.08 7.54 5.69 5.45 5.94 0.12 1.84 0.01 1.58 0.00 1.17
1642207 3.33 31.08 8.23 5.72 5.47 5.97 0.13 2.51 0.01 1.58 0.01 1.59
1645611 6.67 32.08 2.49 6.15 5.99 6.31 0.08 -3.66 0.00 1.58 0.00 -2.32
1680291 10.00 39.08 7.85 6.47 6.22 6.72 0.13 1.38 0.01 1.58 0.00 0.87
1688451 5.00 38.08 6.08 5.83 5.62 6.03 0.11 0.25 0.00 1.58 0.00 0.16
1693207 8.33 27.08 7.15 6.45 6.29 6.61 0.08 0.70 0.00 1.58 0.00 0.44
1768044 6.11 25.08 6.46 6.19 6.05 6.33 0.07 0.27 0.00 1.58 0.00 0.17
1805415 6.11 0.08 5.77 6.62 6.49 6.75 0.06 -0.85 0.00 1.58 0.00 -0.54
1817647 6.11 15.08 7.46 6.36 6.24 6.48 0.06 1.10 0.00 1.58 0.00 0.69
1826491 5.00 43.08 5.77 5.74 5.51 5.97 0.12 0.03 0.01 1.58 0.00 0.02
1831927 5.56 43.08 7.31 5.81 5.60 6.03 0.11 1.49 0.00 1.58 0.00 0.95
1843487 8.33 39.08 6.54 6.25 6.05 6.45 0.10 0.29 0.00 1.58 0.00 0.18
1846887 10.00 35.08 7.15 6.54 6.30 6.78 0.12 0.61 0.01 1.58 0.00 0.39
1884971 7.22 41.08 7.85 6.07 5.88 6.26 0.10 1.78 0.00 1.58 0.00 1.12
1889739 10.00 49.08 7.69 6.30 6.02 6.58 0.14 1.39 0.01 1.58 0.00 0.88
1894487 10.00 33.08 9.23 6.57 6.34 6.81 0.12 2.66 0.01 1.58 0.01 1.68
1955691 7.22 33.08 8.46 6.21 6.04 6.37 0.08 2.26 0.00 1.58 0.00 1.43
1961811 5.56 41.08 6.85 5.85 5.64 6.05 0.10 1.00 0.00 1.58 0.00 0.63
1963167 7.22 34.08 6.15 6.19 6.02 6.35 0.08 -0.03 0.00 1.58 0.00 -0.02
1972699 6.67 -28.92 8.08 7.19 6.97 7.41 0.11 0.89 0.01 1.58 0.00 0.56
2025047 8.33 34.08 5.00 6.34 6.15 6.52 0.09 -1.34 0.00 1.58 0.00 -0.84
2027767 8.89 39.08 7.23 6.32 6.11 6.54 0.11 0.91 0.00 1.58 0.00 0.57
2052927 3.89 38.08 5.54 5.68 5.43 5.92 0.12 -0.14 0.01 1.58 0.00 -0.09
2062451 3.89 27.08 6.23 5.87 5.65 6.08 0.11 0.36 0.00 1.58 0.00 0.23
2069931 5.56 18.08 7.00 6.24 6.10 6.38 0.07 0.76 0.00 1.58 0.00 0.48
2072647 7.78 25.08 4.62 6.41 6.27 6.56 0.07 -1.80 0.00 1.58 0.00 -1.14
2133847 3.89 37.08 3.46 5.70 5.45 5.94 0.12 -2.23 0.01 1.58 0.00 -1.41
2167863 4.44 -29.92 4.62 6.91 6.65 7.17 0.13 -2.30 0.01 1.58 0.01 -1.45
2201171 5.56 -3.92 3.77 6.61 6.47 6.76 0.08 -2.85 0.00 1.58 0.00 -1.80
2202527 10.00 44.08 7.08 6.39 6.12 6.65 0.14 0.69 0.01 1.58 0.00 0.44
2233131 5.00 8.08 2.08 6.34 6.19 6.49 0.08 -4.26 0.00 1.58 0.01 -2.69
2301807 4.44 44.08 7.23 5.65 5.40 5.89 0.12 1.58 0.01 1.58 0.00 1.00
2303847 7.78 38.08 6.00 6.19 6.01 6.38 0.09 -0.19 0.00 1.58 0.00 -0.12
2312007 7.78 -5.92 7.31 6.94 6.80 7.08 0.07 0.36 0.00 1.58 0.00 0.23
2316091 6.67 31.08 6.62 6.17 6.01 6.32 0.08 0.45 0.00 1.58 0.00 0.28
2325615 6.11 -17.92 8.52 6.93 6.75 7.11 0.09 1.59 0.00 1.58 0.00 1.01
2330367 6.11 17.08 6.54 6.33 6.21 6.45 0.06 0.21 0.00 1.58 0.00 0.13
2373887 8.33 40.08 5.92 6.23 6.03 6.44 0.10 -0.31 0.00 1.58 0.00 -0.20
2379327 3.89 44.08 6.85 5.58 5.31 5.84 0.13 1.27 0.01 1.58 0.00 0.80
2419447 6.67 31.08 6.77 6.17 6.01 6.32 0.08 0.60 0.00 1.58 0.00 0.38
2450727 4.44 30.08 6.04 5.89 5.69 6.09 0.10 0.15 0.00 1.58 0.00 0.09
2460931 3.33 22.08 2.46 5.88 5.64 6.11 0.12 -3.42 0.01 1.58 0.01 -2.16
2501047 4.44 40.08 6.23 5.72 5.49 5.95 0.12 0.51 0.01 1.58 0.00 0.32
2587415 8.33 -3.92 4.69 6.98 6.83 7.13 0.08 -2.29 0.00 1.58 0.00 -1.45
2637047 9.44 31.08 5.27 6.53 6.33 6.74 0.11 -1.26 0.00 1.58 0.00 -0.80
2637727 5.00 43.08 5.77 5.74 5.51 5.97 0.12 0.03 0.01 1.58 0.00 0.02
2640451 5.00 -3.92 8.08 6.54 6.38 6.71 0.08 1.54 0.00 1.58 0.00 0.97
2654047 6.67 34.08 7.62 6.11 5.95 6.28 0.08 1.50 0.00 1.58 0.00 0.95
2686011 5.00 17.08 9.08 6.18 6.03 6.34 0.08 2.89 0.00 1.58 0.00 1.83
2749927 10.00 41.08 7.08 6.44 6.18 6.69 0.13 0.64 0.01 1.58 0.00 0.41
2756047 5.00 32.08 5.00 5.93 5.74 6.12 0.10 -0.93 0.00 1.58 0.00 -0.59
2824047 7.22 40.08 7.23 6.09 5.90 6.27 0.10 1.15 0.00 1.58 0.00 0.72
2858047 3.33 20.08 4.00 5.91 5.68 6.14 0.12 -1.91 0.01 1.58 0.00 -1.21
2859411 6.67 26.08 7.00 6.25 6.11 6.39 0.07 0.75 0.00 1.58 0.00 0.47
2894771 3.33 44.08 5.46 5.50 5.22 5.79 0.14 -0.04 0.01 1.58 0.00 -0.03
2938971 6.67 -3.92 2.80 6.76 6.63 6.89 0.07 -3.96 0.00 1.58 0.00 -2.50
2960727 7.22 34.08 4.77 6.19 6.02 6.35 0.08 -1.42 0.00 1.58 0.00 -0.90
2965491 8.33 -3.92 6.76 6.98 6.83 7.13 0.08 -0.22 0.00 1.58 0.00 -0.14
2986567 3.33 39.08 6.62 5.59 5.32 5.86 0.14 1.03 0.01 1.58 0.00 0.65
2990651 8.33 40.08 5.62 6.23 6.03 6.44 0.10 -0.62 0.00 1.58 0.00 -0.39
2996767 5.00 31.08 5.68 5.94 5.76 6.13 0.09 -0.26 0.00 1.58 0.00 -0.17
3023967 5.00 10.08 7.62 6.30 6.15 6.45 0.08 1.31 0.00 1.58 0.00 0.83
3062727 8.33 41.08 7.46 6.22 6.01 6.42 0.11 1.25 0.00 1.58 0.00 0.79
3093327 9.44 22.08 7.31 6.69 6.50 6.87 0.10 0.62 0.00 1.58 0.00 0.39
3115767 8.89 40.08 5.54 6.31 6.09 6.52 0.11 -0.77 0.00 1.58 0.00 -0.49
3123247 6.11 37.08 5.23 5.99 5.81 6.17 0.09 -0.76 0.00 1.58 0.00 -0.48
3197367 5.56 52.08 5.85 5.66 5.41 5.91 0.13 0.19 0.01 1.58 0.00 0.12
3208251 7.22 28.08 7.62 6.29 6.15 6.43 0.07 1.33 0.00 1.58 0.00 0.84
3256527 6.11 40.08 5.69 5.94 5.75 6.13 0.10 -0.25 0.00 1.58 0.00 -0.16
3310927 7.22 21.08 5.38 6.41 6.29 6.53 0.06 -1.02 0.00 1.58 0.00 -0.65
3338127 5.00 21.08 2.00 6.12 5.95 6.28 0.08 -4.12 0.00 1.58 0.01 -2.60
3387771 6.67 27.08 5.38 6.23 6.09 6.37 0.07 -0.85 0.00 1.58 0.00 -0.54
3451007 7.22 38.08 3.77 6.12 5.94 6.30 0.09 -2.35 0.00 1.58 0.00 -1.49
3485687 7.78 15.08 8.38 6.59 6.46 6.71 0.06 1.80 0.00 1.58 0.00 1.14
3495207 10.00 37.08 5.77 6.50 6.26 6.75 0.12 -0.74 0.01 1.58 0.00 -0.47
3504731 7.22 30.08 5.08 6.26 6.11 6.41 0.08 -1.18 0.00 1.58 0.00 -0.75
3527847 5.56 45.08 5.31 5.78 5.56 6.00 0.11 -0.47 0.01 1.58 0.00 -0.30
3555047 5.00 19.08 7.62 6.15 5.99 6.31 0.08 1.47 0.00 1.58 0.00 0.93
3603327 0.83 44.08 7.54 5.17 4.78 5.56 0.20 2.37 0.02 1.58 0.01 1.51
3631887 2.78 29.08 4.54 5.68 5.41 5.96 0.14 -1.15 0.01 1.58 0.00 -0.73
3697167 8.33 35.08 6.08 6.32 6.13 6.50 0.09 -0.24 0.00 1.58 0.00 -0.15
3747491 5.00 19.08 4.08 6.15 5.99 6.31 0.08 -2.07 0.00 1.58 0.00 -1.31
3771287 10.00 14.08 6.10 6.90 6.70 7.10 0.10 -0.80 0.00 1.58 0.00 -0.50
3799851 5.00 34.08 7.54 5.89 5.70 6.09 0.10 1.64 0.00 1.58 0.00 1.04
3803247 3.33 30.08 8.54 5.74 5.49 5.99 0.13 2.80 0.01 1.58 0.01 1.77
3824331 6.67 38.08 2.69 6.05 5.87 6.23 0.09 -3.35 0.00 1.58 0.01 -2.12
3882135 10.00 44.08 5.46 6.39 6.12 6.65 0.14 -0.92 0.01 1.58 0.00 -0.59
3925647 4.44 19.08 4.92 6.08 5.90 6.26 0.09 -1.15 0.00 1.58 0.00 -0.73
3937891 10.00 14.08 6.38 6.90 6.70 7.10 0.10 -0.51 0.00 1.58 0.00 -0.32
4016767 6.67 40.08 6.46 6.01 5.82 6.20 0.10 0.45 0.00 1.58 0.00 0.28
4023571 10.00 34.08 7.83 6.56 6.32 6.79 0.12 1.27 0.01 1.58 0.00 0.81
4046687 8.89 32.08 7.92 6.44 6.25 6.63 0.10 1.48 0.00 1.58 0.00 0.94
4054171 8.89 17.08 8.77 6.70 6.54 6.86 0.08 2.07 0.00 1.58 0.00 1.31
4060291 6.11 24.08 5.23 6.21 6.07 6.35 0.07 -0.98 0.00 1.58 0.00 -0.62
4084767 6.67 29.08 3.31 6.20 6.05 6.35 0.07 -2.89 0.00 1.58 0.00 -1.83
4111287 8.33 37.08 6.00 6.28 6.09 6.48 0.10 -0.28 0.00 1.58 0.00 -0.18
4135087 6.67 36.08 6.31 6.08 5.91 6.25 0.09 0.23 0.00 1.58 0.00 0.14
4149367 5.56 39.08 8.54 5.88 5.68 6.08 0.10 2.66 0.00 1.58 0.00 1.68
4152771 9.44 30.08 6.38 6.55 6.35 6.75 0.10 -0.17 0.00 1.58 0.00 -0.10
4160247 7.78 37.08 7.08 6.21 6.03 6.39 0.09 0.87 0.00 1.58 0.00 0.55
4170447 10.00 -0.92 7.69 7.15 6.95 7.36 0.11 0.54 0.00 1.58 0.00 0.34
4177251 6.67 19.08 4.29 6.37 6.25 6.49 0.06 -2.08 0.00 1.58 0.00 -1.32
4178607 6.67 38.08 8.08 6.05 5.87 6.23 0.09 2.03 0.00 1.58 0.00 1.28
4227567 3.33 35.08 6.77 5.66 5.40 5.91 0.13 1.11 0.01 1.58 0.00 0.71
4229611 4.44 4.08 7.31 6.33 6.16 6.51 0.09 0.98 0.00 1.58 0.00 0.62
4232339 6.11 -28.92 6.92 7.11 6.89 7.34 0.11 -0.19 0.01 1.58 0.00 -0.12
4260891 3.33 12.08 2.85 6.05 5.82 6.27 0.11 -3.20 0.01 1.58 0.01 -2.03
4289447 3.33 40.08 6.00 5.57 5.30 5.84 0.14 0.43 0.01 1.58 0.00 0.27
4293527 10.00 35.08 3.31 6.54 6.30 6.78 0.12 -3.23 0.01 1.58 0.01 -2.05
4311207 6.67 36.08 5.85 6.08 5.91 6.25 0.09 -0.23 0.00 1.58 0.00 -0.15
4313927 8.33 46.08 4.00 6.13 5.90 6.36 0.12 -2.13 0.01 1.58 0.00 -1.35
4366287 6.11 39.08 6.23 5.96 5.77 6.14 0.10 0.28 0.00 1.58 0.00 0.17
4433607 6.67 36.08 5.77 6.08 5.91 6.25 0.09 -0.31 0.00 1.58 0.00 -0.20
4474411 6.67 32.08 6.69 6.15 5.99 6.31 0.08 0.54 0.00 1.58 0.00 0.34
4508411 7.22 41.08 6.92 6.07 5.88 6.26 0.10 0.85 0.00 1.58 0.00 0.54
4543091 7.22 32.08 5.69 6.22 6.06 6.38 0.08 -0.53 0.00 1.58 0.00 -0.33
4570287 9.44 22.08 4.08 6.69 6.50 6.87 0.10 -2.61 0.00 1.58 0.00 -1.65
4609047 6.67 40.08 6.68 6.01 5.82 6.20 0.10 0.66 0.00 1.58 0.00 0.42
4616527 8.33 26.08 5.33 6.47 6.31 6.63 0.08 -1.15 0.00 1.58 0.00 -0.72
4636251 10.00 16.08 3.46 6.86 6.66 7.07 0.10 -3.40 0.00 1.58 0.01 -2.15
4760687 7.22 33.08 8.85 6.21 6.04 6.37 0.08 2.64 0.00 1.58 0.00 1.67
4767487 5.56 8.08 6.62 6.41 6.28 6.54 0.07 0.20 0.00 1.58 0.00 0.13
4768171 5.00 8.08 5.54 6.34 6.19 6.49 0.08 -0.80 0.00 1.58 0.00 -0.50
4815087 5.00 34.08 5.23 5.89 5.70 6.09 0.10 -0.66 0.00 1.58 0.00 -0.42
4823931 8.33 35.08 5.69 6.32 6.13 6.50 0.09 -0.63 0.00 1.58 0.00 -0.40
4825967 4.44 25.08 7.15 5.97 5.78 6.16 0.10 1.18 0.00 1.58 0.00 0.75
4842291 5.56 26.08 5.92 6.10 5.95 6.26 0.08 -0.18 0.00 1.58 0.00 -0.11
4847727 5.56 23.08 4.92 6.15 6.01 6.30 0.08 -1.23 0.00 1.58 0.00 -0.78
4849771 7.22 22.08 4.54 6.39 6.27 6.52 0.06 -1.85 0.00 1.58 0.00 -1.17
4857255 2.78 -5.92 7.77 6.28 6.02 6.55 0.13 1.49 0.01 1.58 0.00 0.94
5427775 7.78 9.08 8.79 6.69 6.57 6.81 0.06 2.10 0.00 1.58 0.00 1.33
5652851 7.78 15.08 5.59 6.59 6.46 6.71 0.06 -0.99 0.00 1.58 0.00 -0.63
6548564 5.56 -20.92 8.08 6.90 6.70 7.11 0.10 1.17 0.00 1.58 0.00 0.74
9288884 8.89 11.08 7.50 6.80 6.65 6.95 0.08 0.70 0.00 1.58 0.00 0.44
11247404 7.78 13.08 2.88 6.62 6.50 6.74 0.06 -3.73 0.00 1.58 0.00 -2.36
11866084 5.56 37.08 7.50 5.92 5.73 6.11 0.10 1.58 0.00 1.58 0.00 1.00
12444084 7.78 19.08 4.62 6.52 6.39 6.64 0.06 -1.90 0.00 1.58 0.00 -1.20
13171764 10.00 13.08 9.23 6.91 6.71 7.12 0.10 2.32 0.00 1.58 0.00 1.47
22596482 7.22 -12.92 6.10 6.99 6.83 7.15 0.08 -0.89 0.00 1.58 0.00 -0.56
22929722 10.00 38.08 7.31 6.49 6.24 6.73 0.13 0.82 0.01 1.58 0.00 0.52
25207642 8.89 30.08 7.25 6.48 6.29 6.66 0.09 0.78 0.00 1.58 0.00 0.49
25316442 6.67 38.08 7.09 6.05 5.87 6.23 0.09 1.04 0.00 1.58 0.00 0.66
25853682 8.33 30.08 3.30 6.40 6.23 6.57 0.09 -3.11 0.00 1.58 0.00 -1.96
25942082 5.56 29.08 3.79 6.05 5.89 6.22 0.08 -2.26 0.00 1.58 0.00 -1.43
26118842 8.89 45.08 7.25 6.22 5.99 6.46 0.12 1.03 0.01 1.58 0.00 0.65
26452122 9.44 -28.92 8.08 7.56 7.30 7.82 0.13 0.52 0.01 1.58 0.00 0.33
26683282 7.22 42.08 3.79 6.05 5.86 6.25 0.10 -2.26 0.00 1.58 0.00 -1.43
26771642 5.56 -0.92 3.92 6.56 6.42 6.71 0.07 -2.64 0.00 1.58 0.00 -1.67
26792082 6.67 34.08 5.38 6.11 5.95 6.28 0.08 -0.73 0.00 1.58 0.00 -0.46
26873642 4.44 -20.92 7.69 6.76 6.53 6.99 0.12 0.93 0.01 1.58 0.00 0.59
27043642 3.89 38.08 5.27 5.68 5.43 5.92 0.12 -0.40 0.01 1.58 0.00 -0.26
27240842 6.67 24.08 6.26 6.28 6.15 6.42 0.07 -0.02 0.00 1.58 0.00 -0.01
27417642 8.33 38.08 6.54 6.27 6.07 6.46 0.10 0.27 0.00 1.58 0.00 0.17
27689642 6.11 44.08 6.10 5.87 5.66 6.08 0.11 0.23 0.00 1.58 0.00 0.14
27873242 5.00 23.08 7.09 6.08 5.92 6.25 0.08 1.01 0.00 1.58 0.00 0.64
27961722 7.22 19.08 7.12 6.44 6.32 6.56 0.06 0.67 0.00 1.58 0.00 0.42
27968482 4.44 42.08 4.95 5.68 5.45 5.92 0.12 -0.74 0.01 1.58 0.00 -0.47
28417242 5.83 32.08 5.93 6.04 5.87 6.21 0.09 -0.10 0.00 1.58 0.00 -0.07
28451242 5.00 34.08 7.42 5.89 5.70 6.09 0.10 1.52 0.00 1.58 0.00 0.96
28458082 5.56 32.08 6.10 6.00 5.83 6.17 0.09 0.10 0.00 1.58 0.00 0.06
29172082 10.00 36.08 6.43 6.52 6.28 6.76 0.12 -0.09 0.01 1.58 0.00 -0.06
30103642 7.78 8.08 3.96 6.70 6.59 6.82 0.06 -2.75 0.00 1.58 0.00 -1.74
30341682 5.00 30.08 5.27 5.96 5.78 6.14 0.09 -0.69 0.00 1.58 0.00 -0.43
30450442 8.33 22.08 5.27 6.54 6.39 6.69 0.07 -1.26 0.00 1.58 0.00 -0.80
30708882 9.44 15.08 8.77 6.81 6.63 6.98 0.09 1.96 0.00 1.58 0.00 1.24
31756082 6.67 26.08 5.93 6.25 6.11 6.39 0.07 -0.32 0.00 1.58 0.00 -0.20
31851242 5.56 37.08 5.60 5.92 5.73 6.11 0.10 -0.31 0.00 1.58 0.00 -0.20
31864842 10.00 37.08 3.30 6.50 6.26 6.75 0.12 -3.21 0.01 1.58 0.01 -2.03
32456482 8.33 19.08 3.85 6.59 6.45 6.73 0.07 -2.74 0.00 1.58 0.00 -1.73
32619722 7.22 -3.92 4.12 6.84 6.71 6.96 0.07 -2.71 0.00 1.58 0.00 -1.72
32626442 3.33 31.08 5.93 5.72 5.47 5.97 0.13 0.21 0.01 1.58 0.00 0.13
32735242 7.22 15.08 9.23 6.51 6.40 6.62 0.06 2.72 0.00 1.58 0.00 1.72
32918842 6.67 -2.92 3.79 6.75 6.62 6.87 0.06 -2.95 0.00 1.58 0.00 -1.87
33265642 6.67 31.08 5.77 6.17 6.01 6.32 0.08 -0.40 0.00 1.58 0.00 -0.25
33741642 6.67 30.08 3.79 6.18 6.03 6.33 0.08 -2.39 0.00 1.58 0.00 -1.51
33816442 9.44 3.08 8.90 7.01 6.83 7.19 0.09 1.89 0.00 1.58 0.00 1.20
33870882 8.33 39.08 5.44 6.25 6.05 6.45 0.10 -0.81 0.00 1.58 0.00 -0.51
33966042 7.22 28.08 5.93 6.29 6.15 6.43 0.07 -0.36 0.00 1.58 0.00 -0.23
34068042 6.11 30.08 7.25 6.11 5.95 6.26 0.08 1.14 0.00 1.58 0.00 0.72
34387642 3.33 25.08 3.63 5.83 5.59 6.06 0.12 -2.20 0.01 1.58 0.00 -1.39
34544042 3.33 27.08 6.15 5.79 5.55 6.03 0.12 0.36 0.01 1.58 0.00 0.23
34571242 6.67 36.08 5.60 6.08 5.91 6.25 0.09 -0.48 0.00 1.58 0.00 -0.30
34748042 6.67 15.08 5.27 6.44 6.33 6.55 0.06 -1.16 0.00 1.58 0.00 -0.74
34754842 8.33 32.08 5.44 6.37 6.19 6.54 0.09 -0.93 0.00 1.58 0.00 -0.59
35026882 6.67 40.08 5.93 6.01 5.82 6.20 0.10 -0.08 0.00 1.58 0.00 -0.05
35108442 7.22 19.08 3.96 6.44 6.32 6.56 0.06 -2.49 0.00 1.58 0.00 -1.57
35271642 8.33 6.08 3.79 6.81 6.68 6.95 0.07 -3.02 0.00 1.58 0.00 -1.91
35666042 6.67 41.08 5.77 6.00 5.80 6.19 0.10 -0.23 0.00 1.58 0.00 -0.14
35672842 6.67 40.08 5.77 6.01 5.82 6.20 0.10 -0.24 0.00 1.58 0.00 -0.15
36577242 5.00 16.08 5.11 6.20 6.05 6.36 0.08 -1.09 0.00 1.58 0.00 -0.69
36822042 10.00 25.08 8.08 6.71 6.49 6.92 0.11 1.37 0.00 1.58 0.00 0.87
36985242 10.00 27.08 5.27 6.68 6.46 6.89 0.11 -1.40 0.00 1.58 0.00 -0.89
37250442 6.11 -1.92 7.09 6.65 6.52 6.78 0.07 0.43 0.00 1.58 0.00 0.27
37495282 5.56 20.08 7.91 6.21 6.06 6.35 0.07 1.71 0.00 1.58 0.00 1.08
37916842 7.22 40.08 7.42 6.09 5.90 6.27 0.10 1.33 0.00 1.58 0.00 0.84
38474442 6.11 22.08 6.43 6.25 6.11 6.38 0.07 0.18 0.00 1.58 0.00 0.12
38488082 10.00 12.08 5.27 6.93 6.73 7.13 0.10 -1.66 0.00 1.58 0.00 -1.05
38712442 5.56 20.08 3.46 6.21 6.06 6.35 0.07 -2.74 0.00 1.58 0.00 -1.73
38766842 7.78 33.08 4.45 6.28 6.11 6.45 0.09 -1.83 0.00 1.58 0.00 -1.16
38964042 6.67 25.08 4.29 6.27 6.13 6.40 0.07 -1.98 0.00 1.58 0.00 -1.25
39032042 3.33 28.08 8.24 5.78 5.53 6.02 0.12 2.47 0.01 1.58 0.01 1.56
39657642 5.00 35.08 6.43 5.88 5.68 6.07 0.10 0.55 0.00 1.58 0.00 0.35
39678042 5.00 36.08 5.11 5.86 5.66 6.06 0.10 -0.75 0.00 1.58 0.00 -0.47
40126842 10.00 33.08 4.95 6.57 6.34 6.81 0.12 -1.63 0.01 1.58 0.00 -1.03
40269642 5.56 33.08 6.10 5.98 5.81 6.16 0.09 0.11 0.00 1.58 0.00 0.07
40392042 7.22 31.08 6.59 6.24 6.09 6.39 0.08 0.35 0.00 1.58 0.00 0.22
40432842 7.22 23.08 8.24 6.38 6.25 6.50 0.07 1.87 0.00 1.58 0.00 1.18
40439642 4.44 28.08 5.11 5.92 5.73 6.12 0.10 -0.81 0.00 1.58 0.00 -0.51
40589282 7.78 15.08 6.10 6.59 6.46 6.71 0.06 -0.49 0.00 1.58 0.00 -0.31
40602842 1.67 9.08 2.31 5.88 5.57 6.19 0.16 -3.57 0.01 1.58 0.02 -2.27
40623242 0.00 40.08 7.09 5.13 4.71 5.55 0.22 1.96 0.02 1.58 0.01 1.25
41065282 3.89 22.08 3.79 5.95 5.74 6.16 0.11 -2.16 0.00 1.58 0.00 -1.37
41357642 7.22 23.08 3.30 6.38 6.25 6.50 0.07 -3.08 0.00 1.58 0.00 -1.95
41364442 6.67 32.08 5.77 6.15 5.99 6.31 0.08 -0.38 0.00 1.58 0.00 -0.24
41595642 7.22 32.08 9.23 6.22 6.06 6.38 0.08 3.01 0.00 1.58 0.00 1.90
41745242 8.33 35.08 8.24 6.32 6.13 6.50 0.09 1.92 0.00 1.58 0.00 1.22
41752042 10.00 47.08 9.23 6.33 6.06 6.61 0.14 2.90 0.01 1.58 0.01 1.84
42187242 4.44 32.08 6.43 5.85 5.65 6.06 0.11 0.57 0.00 1.58 0.00 0.36
42602042 3.89 53.08 5.93 5.42 5.13 5.72 0.15 0.51 0.01 1.58 0.00 0.32
42656442 7.78 26.08 7.42 6.40 6.25 6.54 0.07 1.02 0.00 1.58 0.00 0.64
42670042 10.00 15.08 6.26 6.88 6.68 7.08 0.10 -0.62 0.00 1.58 0.00 -0.39
42690442 6.67 35.08 6.92 6.10 5.93 6.27 0.09 0.83 0.00 1.58 0.00 0.52
42778842 7.78 22.08 8.57 6.47 6.33 6.60 0.07 2.11 0.00 1.58 0.00 1.33
42812882 5.00 38.08 6.59 5.83 5.62 6.03 0.11 0.77 0.00 1.58 0.00 0.49
42914842 9.17 37.08 2.64 6.39 6.18 6.61 0.11 -3.76 0.00 1.58 0.01 -2.38
42962442 7.22 8.08 4.23 6.63 6.52 6.74 0.06 -2.40 0.00 1.58 0.00 -1.52
43275282 7.22 27.08 6.73 6.31 6.17 6.45 0.07 0.42 0.00 1.58 0.00 0.27
43418082 3.33 30.08 9.07 5.74 5.49 5.99 0.13 3.32 0.01 1.58 0.01 2.11
47899242 4.44 40.08 5.96 5.72 5.49 5.95 0.12 0.24 0.01 1.58 0.00 0.15
49198042 6.11 8.08 4.62 6.48 6.37 6.60 0.06 -1.87 0.00 1.58 0.00 -1.18
49592442 10.00 20.08 3.27 6.79 6.59 7.00 0.11 -3.53 0.00 1.58 0.01 -2.23
52162882 3.89 -3.92 6.23 6.39 6.18 6.60 0.11 -0.16 0.00 1.58 0.00 -0.10
54032962 7.78 12.08 7.15 6.64 6.52 6.75 0.06 0.52 0.00 1.58 0.00 0.33
54400042 8.89 23.08 5.31 6.60 6.43 6.76 0.09 -1.29 0.00 1.58 0.00 -0.81
54434082 6.11 23.08 6.00 6.23 6.09 6.36 0.07 -0.23 0.00 1.58 0.00 -0.14
54644882 8.89 28.08 7.38 6.51 6.33 6.69 0.09 0.87 0.00 1.58 0.00 0.55
54760442 10.00 20.08 5.08 6.79 6.59 7.00 0.11 -1.72 0.00 1.58 0.00 -1.09
55100442 4.44 -5.92 3.92 6.50 6.31 6.69 0.10 -2.58 0.00 1.58 0.00 -1.63
55420042 7.22 32.08 7.15 6.22 6.06 6.38 0.08 0.93 0.00 1.58 0.00 0.59
55569642 8.89 31.08 6.69 6.46 6.27 6.65 0.10 0.23 0.00 1.58 0.00 0.15
56385642 5.00 25.08 3.92 6.05 5.88 6.22 0.09 -2.12 0.00 1.58 0.00 -1.34
56487642 7.78 38.08 6.00 6.19 6.01 6.38 0.09 -0.19 0.00 1.58 0.00 -0.12
56841242 10.00 38.08 4.62 6.49 6.24 6.73 0.13 -1.87 0.01 1.58 0.00 -1.19
56882042 7.22 28.08 7.38 6.29 6.15 6.43 0.07 1.09 0.00 1.58 0.00 0.69
57140442 5.00 20.08 5.54 6.13 5.97 6.29 0.08 -0.59 0.00 1.58 0.00 -0.38
57201642 5.00 17.08 4.33 6.18 6.03 6.34 0.08 -1.86 0.00 1.58 0.00 -1.17
57256082 5.00 5.08 5.54 6.39 6.24 6.54 0.08 -0.85 0.00 1.58 0.00 -0.54
57630042 7.78 30.08 2.77 6.33 6.17 6.49 0.08 -3.56 0.00 1.58 0.00 -2.25
57636842 5.56 31.08 5.08 6.02 5.85 6.19 0.09 -0.94 0.00 1.58 0.00 -0.60
57677642 6.67 35.08 5.31 6.10 5.93 6.27 0.09 -0.79 0.00 1.58 0.00 -0.50
58085642 8.33 46.08 7.38 6.13 5.90 6.36 0.12 1.25 0.01 1.58 0.00 0.79
58371282 10.00 20.08 1.85 6.79 6.59 7.00 0.11 -4.95 0.00 1.58 0.01 -3.13
58616042 8.33 32.08 6.69 6.37 6.19 6.54 0.09 0.32 0.00 1.58 0.00 0.20
59316442 10.00 30.08 6.92 6.62 6.40 6.85 0.12 0.30 0.01 1.58 0.00 0.19
59819642 4.44 32.08 4.62 5.85 5.65 6.06 0.11 -1.24 0.00 1.58 0.00 -0.78
59833242 7.22 33.08 7.15 6.21 6.04 6.37 0.08 0.95 0.00 1.58 0.00 0.60
60010042 3.33 26.08 3.92 5.81 5.57 6.05 0.12 -1.89 0.01 1.58 0.00 -1.19
60146042 6.67 44.08 4.62 5.94 5.74 6.15 0.10 -1.33 0.00 1.58 0.00 -0.84
60384042 5.00 30.08 3.92 5.96 5.78 6.14 0.09 -2.04 0.00 1.58 0.00 -1.29
60941642 6.11 34.08 8.31 6.04 5.87 6.21 0.09 2.27 0.00 1.58 0.00 1.43
61512842 6.67 25.08 5.54 6.27 6.13 6.40 0.07 -0.73 0.00 1.58 0.00 -0.46
61669242 9.44 -5.92 5.54 7.16 6.97 7.36 0.10 -1.63 0.00 1.58 0.00 -1.03
62002482 10.00 25.08 6.06 6.71 6.49 6.92 0.11 -0.65 0.00 1.58 0.00 -0.41
62689242 10.00 35.08 3.92 6.54 6.30 6.78 0.12 -2.62 0.01 1.58 0.01 -1.66
62859242 9.44 13.08 6.46 6.84 6.66 7.02 0.09 -0.38 0.00 1.58 0.00 -0.24
62913642 9.44 30.08 3.46 6.55 6.35 6.75 0.10 -3.09 0.00 1.58 0.01 -1.95
63049642 7.78 -1.92 5.54 6.88 6.74 7.01 0.07 -1.34 0.00 1.58 0.00 -0.84
63124442 2.78 34.08 6.00 5.60 5.32 5.88 0.14 0.40 0.01 1.58 0.00 0.25
63410042 10.00 25.08 7.38 6.71 6.49 6.92 0.11 0.68 0.00 1.58 0.00 0.43
63580042 10.00 20.08 2.77 6.79 6.59 7.00 0.11 -4.03 0.00 1.58 0.01 -2.55
64022082 5.00 15.08 3.46 6.22 6.06 6.37 0.08 -2.76 0.00 1.58 0.00 -1.74
64328042 7.78 -0.92 6.00 6.86 6.73 6.99 0.07 -0.86 0.00 1.58 0.00 -0.54
64749642 2.78 -1.92 2.77 6.21 5.95 6.47 0.13 -3.44 0.01 1.58 0.01 -2.18
64797242 8.33 25.08 5.54 6.49 6.33 6.64 0.08 -0.95 0.00 1.58 0.00 -0.60
64865242 5.56 33.08 2.77 5.98 5.81 6.16 0.09 -3.22 0.00 1.58 0.00 -2.03
64872082 8.33 40.08 6.23 6.23 6.03 6.44 0.10 0.00 0.00 1.58 0.00 0.00
64878842 5.00 30.08 3.92 5.96 5.78 6.14 0.09 -2.04 0.00 1.58 0.00 -1.29
65314042 7.78 17.08 4.15 6.55 6.43 6.67 0.06 -2.40 0.00 1.58 0.00 -1.52
65504442 7.78 31.08 5.31 6.31 6.15 6.47 0.08 -1.01 0.00 1.58 0.00 -0.64
66327242 6.11 6.08 5.77 6.52 6.40 6.64 0.06 -0.75 0.00 1.58 0.00 -0.47
66408842 5.00 29.08 6.23 5.98 5.80 6.16 0.09 0.25 0.00 1.58 0.00 0.16
66653642 6.11 18.08 5.08 6.31 6.19 6.44 0.06 -1.24 0.00 1.58 0.00 -0.78
66755642 5.00 34.08 8.77 5.89 5.70 6.09 0.10 2.88 0.00 1.58 0.00 1.82
66857642 10.00 18.08 5.08 6.83 6.62 7.03 0.10 -1.75 0.00 1.58 0.00 -1.11
66891642 7.22 25.08 3.46 6.34 6.21 6.48 0.07 -2.88 0.00 1.58 0.00 -1.82
67020842 8.33 24.08 7.38 6.51 6.35 6.66 0.08 0.88 0.00 1.58 0.00 0.56
67136442 2.78 35.08 5.31 5.58 5.30 5.87 0.14 -0.27 0.01 1.58 0.00 -0.17
67190842 6.67 25.08 2.54 6.27 6.13 6.40 0.07 -3.73 0.00 1.58 0.00 -2.36
67252042 3.89 16.08 3.00 6.05 5.85 6.25 0.10 -3.05 0.00 1.58 0.01 -1.93
67381322 6.11 -27.92 7.31 7.10 6.88 7.32 0.11 0.21 0.01 1.58 0.00 0.13
67666922 3.89 38.08 6.23 5.68 5.43 5.92 0.12 0.55 0.01 1.58 0.00 0.35

Models + broom::augment()

  • One standard diagnostic plot is to plot fitted values v residuals
  • Looks a little wonky (remember, these are results from multiple harmonized studies)
nested_lm %>%
  select(study, data) %>%
  unnest(data) %>%
  ggplot(aes(
    x = .fitted
    , y = .resid
  )) + 
  geom_point() + 
  theme_classic()

Models + broom::augment()

  • One standard diagnostic plot is to plot fitted values v residuals
  • Looks a little wonky (remember, these are results from multiple harmonized studies)
nested_lm %>%
  select(study, data) %>%
  unnest(data) %>%
  ggplot(aes(
    x = .fitted
    , y = .resid
  )) + 
  geom_point() +
  labs(
    x = "Model Fitted Values"
    , y = "Residual") + 
  facet_wrap(~study) + 
  theme_classic()

Models + broom::augment()

Another is raw v. fitted

nested_lm %>%
  select(study, data) %>%
  unnest(data) %>%
  ggplot(aes(
    x = p_value
    , y = .resid
  )) + 
  geom_point() +
  facet_wrap(~study) + 
  theme_classic()

Model Predictions

While augment gives you the person-level predictions, what it doesn’t do is give you nice smooth trajectories. This is because the persons have different levels of covariates. Thus, augment is not ideal for plotting predictions unless (1) there are no covariates or (2) all covariates are mean-centered.

nested_lm %>%
  select(study, data) %>%
  unnest(data) %>%
  ggplot(aes(
    x = p_value
    , y = .fitted
  )) + 
  geom_point(alpha = .5, color = "grey") + 
  geom_line() +
  facet_wrap(~study) + 
  theme_classic()

Model Predictions

  • Although we can get the standard error of the prediction for each person, we often want to look at theoretical predictions, adjusting for covariates. We can typically use built-in predict() or fitted() functions
  • To do this, we need to see theoretical ranges of key variables and grab averages of covariates
  • I use functions for this. We’ll do one without a function and build there
m1 <- nested_lm$m[[1]]
d1 <- m1$model

crossing(
  p_value = seq(0, 10, length.out = 100)
  , age = mean(d1$age)
) %>%
  bind_cols(
    .
    , predict(m1, newdata = ., interval = "prediction")
  )
# A tibble: 100 × 5
   p_value   age   fit   lwr   upr
     <dbl> <dbl> <dbl> <dbl> <dbl>
 1   0      9.42  5.65  2.52  8.79
 2   0.101  9.42  5.67  2.53  8.80
 3   0.202  9.42  5.68  2.55  8.81
 4   0.303  9.42  5.69  2.56  8.82
 5   0.404  9.42  5.71  2.57  8.84
 6   0.505  9.42  5.72  2.59  8.85
 7   0.606  9.42  5.73  2.60  8.86
 8   0.707  9.42  5.75  2.62  8.87
 9   0.808  9.42  5.76  2.63  8.89
10   0.909  9.42  5.77  2.64  8.90
# ℹ 90 more rows

Model Predictions

Prediction Interval

crossing(
  p_value = seq(0, 10, .1)
  , age = mean(d1$age)
) %>%
  bind_cols(
    .
    , predict(m1, newdata = ., interval = "prediction")
  ) %>%
  ggplot(aes(x = p_value, y = fit)) + 
    geom_ribbon(aes(ymin = lwr, ymax = upr), fill = "seagreen4", alpha = .2) + 
    geom_line(color = "seagreen4", size = 2) + 
    theme_classic()

Model Predictions

Confidence Interval

crossing(
  p_value = seq(0, 10, .1)
  , age = mean(d1$age)
) %>%
  bind_cols(
    .
    , predict(m1, newdata = ., interval = "confidence")
  ) %>%
  ggplot(aes(x = p_value, y = fit)) + 
    geom_ribbon(aes(ymin = lwr, ymax = upr), fill = "seagreen4", alpha = .2) + 
    geom_line(color = "seagreen4", size = 2) + 
    theme_classic()

Model Predictions

  • This is fine, but it could use some improvements:
    • better scales
    • raw data
    • the usual aesthetics

Model Predictions

Better scales

crossing(
  p_value = seq(0, 10, .1)
  , age = mean(d1$age)
) %>%
  bind_cols(
    .
    , predict(m1, newdata = ., interval = "prediction")
  ) %>%
  ggplot(aes(x = p_value, y = fit)) + 
    geom_ribbon(aes(ymin = lwr, ymax = upr), fill = "seagreen4", alpha = .2) + 
    geom_line(color = "seagreen4", size = 2) + 
    scale_x_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    scale_y_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    theme_classic()

Model Predictions

Raw Data

crossing(
  p_value = seq(0, 10, .1)
  , age = mean(d1$age)
) %>%
  bind_cols(., predict(m1, newdata = ., interval = "prediction")) %>%
  ggplot(aes(x = p_value, y = fit)) + 
    geom_point(
      data = d1
      , aes(x = p_value, y = SRhealth)
      , alpha = .4
      , color = "seagreen4"
      ) + 
    geom_ribbon(aes(ymin = lwr, ymax = upr), fill = "seagreen4", alpha = .2) + 
    geom_line(color = "seagreen4", size = 2) + 
    scale_x_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    scale_y_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    theme_classic()

Model Predictions

The usual aesthetics

crossing(
  p_value = seq(0, 10, .1)
  , age = mean(d1$age)
) %>%
  bind_cols(., predict(m1, newdata = ., interval = "prediction")) %>%
  ggplot(aes(x = p_value, y = fit)) + 
    geom_point(data = d1, aes(x = p_value, y = SRhealth)
      , alpha = .4, color = "seagreen4") + 
    geom_ribbon(aes(ymin = lwr, ymax = upr), fill = "seagreen4", alpha = .2) + 
    geom_line(color = "seagreen4", size = 2) + 
    scale_x_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    scale_y_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    labs(
      x = "Conscientiousness (POMP; 0-10)"
      , y = "Predicted Self-Rated Health (POMP; 0-10)"
      , title = "Conscientiousness and Self-Rated Health\nWere Weakly Associated"
      ) + 
    theme_classic() + 
    theme(
      axis.text = element_text(face = "bold", size = rel(1.1))
      , axis.title = element_text(face = "bold", size = rel(1.1))
      , plot.title = element_text(face = "bold", size = rel(1.2), hjust = .5)
      )

Model Predictions

Across All Samples

pred_fun <- function(m){
  d <- m$model

  crossing(
    p_value = seq(0, 10, length.out = 100)
    , age = mean(d$age)
  ) %>%
    bind_cols(
      .
      , predict(m, newdata = ., interval = "prediction")
    )
}
nested_lm <- nested_lm %>%
  mutate(pred = map(m, pred_fun))
nested_lm
# A tibble: 6 × 6
  study  data                  m      tidy             glance   pred    
  <chr>  <list>                <list> <list>           <list>   <list>  
1 Study1 <tibble [831 × 13]>   <lm>   <tibble [3 × 7]> <tibble> <tibble>
2 Study2 <tibble [996 × 13]>   <lm>   <tibble [3 × 7]> <tibble> <tibble>
3 Study3 <tibble [1,000 × 13]> <lm>   <tibble [3 × 7]> <tibble> <tibble>
4 Study4 <tibble [574 × 13]>   <lm>   <tibble [3 × 7]> <tibble> <tibble>
5 Study5 <tibble [616 × 13]>   <lm>   <tibble [3 × 7]> <tibble> <tibble>
6 Study6 <tibble [1,000 × 13]> <lm>   <tibble [3 × 7]> <tibble> <tibble>

Model Predictions

Across All Samples

nested_lm %>%
  select(study, pred) %>%
  unnest(pred)
# A tibble: 600 × 6
   study  p_value   age   fit   lwr   upr
   <chr>    <dbl> <dbl> <dbl> <dbl> <dbl>
 1 Study1   0      9.42  5.65  2.52  8.79
 2 Study1   0.101  9.42  5.67  2.53  8.80
 3 Study1   0.202  9.42  5.68  2.55  8.81
 4 Study1   0.303  9.42  5.69  2.56  8.82
 5 Study1   0.404  9.42  5.71  2.57  8.84
 6 Study1   0.505  9.42  5.72  2.59  8.85
 7 Study1   0.606  9.42  5.73  2.60  8.86
 8 Study1   0.707  9.42  5.75  2.62  8.87
 9 Study1   0.808  9.42  5.76  2.63  8.89
10 Study1   0.909  9.42  5.77  2.64  8.90
# ℹ 590 more rows

Model Predictions

Very close, but our intervals are cut off

nested_lm %>%
  select(study, pred) %>%
  unnest(pred) %>%
  ggplot(aes(x = p_value, y = fit)) + 
    geom_point(data = d1, aes(x = p_value, y = SRhealth)
      , alpha = .2, color = "seagreen4") + 
    geom_ribbon(aes(ymin = lwr, ymax = upr), fill = "seagreen4", alpha = .2) + 
    geom_line(color = "seagreen4", size = 2) + 
    scale_x_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    scale_y_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    labs(
      x = "Conscientiousness (POMP; 0-10)"
      , y = "Predicted Self-Rated Health (POMP; 0-10)"
      , title = "Conscientiousness and Self-Rated Health\nWere Weakly Associated In Most Samples"
      ) + 
    facet_wrap(~study, ncol = 2) + 
    theme_classic() + 
    theme(
      axis.text = element_text(face = "bold", size = rel(1.1))
      , axis.title = element_text(face = "bold", size = rel(1.1))
      , plot.title = element_text(face = "bold", size = rel(1.2), hjust = .5)
      , strip.background = element_rect(fill = "darkseagreen4")
      , strip.text = element_text(face = "bold", color = "white")
      )

Model Predictions

nested_lm %>%
  select(study, pred) %>%
  unnest(pred) %>%
  mutate(upr = ifelse(upr > 10, 10, upr)
         , lwr = ifelse(lwr < 0, 0, lwr)) %>%
  ggplot(aes(x = p_value, y = fit)) + 
    geom_point(data = d1, aes(x = p_value, y = SRhealth)
      , alpha = .2, color = "seagreen4") + 
    geom_ribbon(aes(ymin = lwr, ymax = upr), fill = "seagreen4", alpha = .2) + 
    geom_line(color = "seagreen4", size = 2) + 
    scale_x_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    scale_y_continuous(limits = c(0,10.2), breaks = seq(0,10,2)) + 
    labs(
      x = "Conscientiousness (POMP; 0-10)"
      , y = "Predicted Self-Rated Health (POMP; 0-10)"
      , title = "Conscientiousness and Self-Rated Health\nWere Weakly Associated In Most Samples"
      ) + 
    facet_wrap(~study, ncol = 2) + 
    theme_classic() + 
    theme(
      axis.text = element_text(face = "bold", size = rel(1.1))
      , axis.title = element_text(face = "bold", size = rel(1.1))
      , plot.title = element_text(face = "bold", size = rel(1.2), hjust = .5)
      , strip.background = element_rect(fill = "darkseagreen4")
      , strip.text = element_text(face = "bold", color = "white")
      )

Wrapping Up

  • This is a quick introduction to visualizing associations and working with models
  • Here, we focused on doing things very manually to promote understanding
  • But there are lots of packages to automate much of this:

PSC 290 - Data Visualization