Ask MetaFilter questions tagged with r

Rates of success?

rollick — Wed, 04 Nov 2009 09:46:36 -0800

Statistics question: is it possible to test sets of cumulative data for significant differences in rate? I have three cumulative percentage graphs, measuring the germination rates of three different seed types. Is there a way to compare them and see if there are any statistically significant differences?

The seed types were planted in triplicate, on three dishes each (nine overall). Every day for the past few weeks I've observed how many seeds on each dish have begun germinating -- so for an individual dish I would have " Day 1: 0 ... Day 7: 14 ... Day 14: 29" etc, with each day's score a cumulative total. (There are 100 seeds on each dish, so it works as a percentage rate as well)

In Excel, I've graphed the average germination rates of the replicates, for a graph that resembles this one. (with three lines plotted, and x-axis = time in days, y-axis = percent germinated).

So is there a way to compare these different rates statistically? I can use Excel, Minitab, SPSS, and R.

BookSuggestionFilter: I need to learn R (and about statistics) in a hurry.

kenliu — Thu, 22 Oct 2009 19:17:29 -0800

BookSuggestionFilter: I need to learn about R (and statistical modeling) in a hurry. I need to learn R in a hurry. I'm heading full steam into a new software development project at work where I will be working with some hard-core statistical modelers who are building models in R (as well as using other stuff like SAS).

I have a degree in engineering so I am ok with the math, but it has been over a decade since I did anything with statistics (and it was pretty rough even back then).

I'm looking for some book recommendations for the following:
1) a refresher on general statistics and some introductory material with statistical modeling
2) a book about R for someone who is an experienced software developer

[R] you experienced with plot() and barplot() ?

bengarland — Thu, 09 Apr 2009 16:39:39 -0800

How do I graph these data using R's plot() and barplot() commands? I have data that I need to graph using R. Unfortunately, the examples I have found in books and on the web are too simplistic and do not give me much insight into the process. I have never used R before, and have been trying for hours and hours to figure out how to do what I need to do. I also already read the previous posts about R here on Ask MeFi, but I am still stumped.

I've made a fake set of data that represents the structure of my real data (which you don't want to see, it's too confusing).

Here is my sample data.

Take a quick look at that.

There is a main factor/category, Factor1 with values of "BOB" and "JILL", and a grouping within that called SubFactor with values "Q" and "P". I also have blocked reps of everything, labeled A, B, C, D. My measurements are Measurement1, Measurement2, Measurement3, etc which I will abbreviate as M1, M2, M3, etc.

To cut to the chase, I made a few hand drawings of the graphs I need. Please note that the graphs are illustrative only, and do not correspond to the data above.

Please walk me through this step-by-step and assume I know absolutely nothing about how to get my data into R, how to manipulate it as needed, and finally, how to produce the following graphs.

ALSO if any of you R graphing experts want to take me under your wing and help me learn more than I have outlined here (surely I will have future questions), please send me a MeMail.

----------

GRAPH 1...

In the first, I want to make a bar graph of just the average of the M1 measurements for each treatment, split into categories "BOB" and "JILL" with the grouped "P" and "Q" subfactors.

So that we're on the same page here, the mean of BOB(Q) for M1 = (100+500+200+300)/4 = 275, for example.

Here is my drawing for this: Bar Graph 1

----------

GRAPH 2...

In the second, I want a plot of the measurements for "BOB", with M1, M2, M3, M4, and M5 on the X-axis. The lines will represent the "P" and "Q" subfactors.

Here is my drawing for this: Plot 1

----------

GRAPH 3...

In another bar graph, I want to have M1, M2, M3, M4, M5 be the bars, and have them grouped by BOB (Q, P) and JILL (Q, P) on the x-axis. The Y-axis would show the means.

Here is my drawing for this: Bar Graph 2

----------

GRAPH 4...

In the final plot, I want to graph only the mean measurements for "BOB" sub-level "P" over time. Let's say that each M1, M2, M3, M4, M5 was taken at unique times T1, T2, T3, T4, T5 where the times could have values T1=15, T2=45, T3=55, T4=65, T5=70 and they need to be spaced properly on a timeline (i.e. the distance between T1 and T2 should be 30 units, T2 and T3 10 units, T3 and T4 10 units, T4 and T5 5 units) -- I spaced them evenly on the graph, so ignore that.

Here is my drawing for this: Plot 2

----------

THANKS!!!

R.rrrrgh

chrisalbon — Sat, 14 Mar 2009 15:06:04 -0800

Are There Good Blogs About The Statistical Programming Language, R? I recently found the R Cookbook site and have enjoyed the R code and tips offered. I was wondering if there are other blogs out there that discuss how to use R better.

P.S. I am asking this on ask.metafilter because Googling "R" is supremely unhealthful.

How can I convert a SAS dataset into something readable by R?

docgonzo — Fri, 23 Jan 2009 17:48:52 -0800

How to import a SAS dataset into R (with, unfortunately, one extra degree of difficulty...)? I have a sas7bdat file. I'd like to import it into R (OS X). Unfortunately, I don't have access to SAS -- I'm away from work, and the file's sender has gone home for the weekend. Any free means of converting the file into a format readable by R? Thanks!

Elegant weapons, for a more civilized age

ghost of a past number — Sat, 25 Oct 2008 02:35:07 -0800

Do lisp and dialects make sense for general purpose scientific computing? I'm thinking of learning some variant of lisp as my next language, mostly out of masochism. My real-world computing needs are scientific/numerical, i.e. data manipulation, some statistics, lots of curve fitting and the like, with some data acquisition thrown in . So far I have been using C for the heavy stuff, perl for the quick and dirty and FORTRAN when I have to (I hang around with engineers).

Is lisp any use for such things? All the functional recursiveness seems pretty nifty, but I'd like to pick up a tool I can actually use.

Visually exploring and representing survey data.

proj — Wed, 07 May 2008 09:45:08 -0800

Best ways to visually explore a large survey data set? My advisor has advised me to explore my data set visually before diving in statistically. It is a large (N = 180,000+) survey data set comprised of individuals in over 80 countries. Most of the responses are categorical or dichotomous in nature, taking the form "agree/disagree" or "yes/no/maybe." Some of them are Likert-style scales (1-5, Disagree-Agree). Many of the demographic variables are also categorical (for example, rather than asking income, "income level" is asked) but I do have a few continuous variables such as age. My dependent variable of interest is a scale composed of four survey items indexed to 100 (although the actual number of discrete values taken on the scale is rather low owing to the nature of the questions comprising the scale).

What would be some interesting ways to visually explore this data? Obviously, scatterplots (even with jittering) are not the way to go because of the highly redundant and categorical nature of the data. I have a few boxplots that I've generated (usually separating by gender or region). I am open to abstract suggestions or concrete suggestions using R or Stata.

RPy problems

Blazecock Pileon — Mon, 25 Feb 2008 21:05:44 -0800

Stumped with RPy, need help badly! I'd like to use RPy to try to manipulate a Python 2.3.4 list within R 2.4.1. My list is made up of five arrays (four of string-type — "utility", "target", "build" and "timeType" — and one float-type — "time").

My problem is that I can't seem to build a data frame in R. I'd like to, for example, group my data analysis by 'utility, 'target' and 'build' from calls made within Python.

When I use r.data_frame() to create a data frame object, the resulting object is not an R data frame. The following prints "False" on the call from r.is_data_frame():

==========

timeDataFrame = { "utility":[],

                  "target":[],

                  "build":[],

                  "timeType":[],

                  "time":[] }



for timeDataListObj in timeDataListArray:

  for timeDataObj in timeDataListObj.timedata:

    for timeDataType in timeDataTypes:

      timeDataFrame["utility"].append(timeDataListObj.utility)

      timeDataFrame["target"].append(timeDataListObj.target)

      timeDataFrame["build"].append(timeDataListObj.build)

      timeDataFrame["timeType"].append(timeDataType)

      timeDataFrame["time"].append(float(timeValue))



df = r.data_frame(timeDataFrame["utility"], \

             timeDataFrame["target"], \

             timeDataFrame["build"], \

             timeDataFrame["timeType"], \

             timeDataFrame["time"])

r.print_(r.is_data_frame(df))

==========

Another problem I have is with syntax. For example, how can I perform a column reference like df$target or df$timeType?

When I tried to do either:

r.print_(df$target)

or

r.print_(df+r['$']+target)

or

r['print(df$target)']

I get syntax errors. Same with r.split(df$target, df$build) and similar.

The problem seems to come down to how these are interpreted. Either Python misinterprets the r.print_() calls and complains about the $ reference, or when I use r['print(df$target)'], the R interpreter doesn't have any knowledge of the variable df and complains about non-existent variables.

Any advice from seasoned Python/R/RPy users would be greatly appreciated. Thanks!

R

chrisalbon — Tue, 12 Feb 2008 16:58:14 -0800

I am looking for great books / guides for learning applied statistics and R (at the same time). Any hivemind suggestions? I am a graduate student in political science. This summer I have 114 days of no work, no research, and no classes (yeah!). Along with studying for my comprehensive exams, I want to spend this time getting a good handle on applied statistics and the program "R".

Little Background:

I have weak background in mathematics but a strong background (and proficiency) in programming.

My first statistics classes were heavily "math / theory" oriented (proofs, formulas, etc...). I had trouble grasping it and really did not enjoy myself.

However, in the more advanced classes we started using R and I loved it. To me, statistics in R is just enough programming language to learn / tinker in, something I am very comfortable with. I throughly enjoy spending time hacking code and so learning statistics through R seems like an interesting idea.

This is all just a long winded way to asking: Does anyone knows of some great introductory, intermediate, or advanced books that teach applied statistics through using R?

What is this symbol that looks like an O with a small n or r in it?

i love cheese — Fri, 02 Nov 2007 08:54:23 -0800

What is this logical/matrix/algorithm symbol? I'm trying to implement an algorithm for manipulating matrices and I think I understand the equation described, but I don't know what the symbol that looks like a big O with a small n or r in it is called, or what it really means.

The actual computation seems to be a calculation of Minkowski distance having exactly i+1 links, and that it relies on knowing the distance having i links. I've never seen that symbol before, and would like to know what it means

Optimal output settings when generating images for use in PowerPoint

grouse — Mon, 15 Oct 2007 05:22:53 -0800

What are the optimal output settings when generating images for use in PowerPoint? I am producing plots for use in PowerPoint in R. I tried using Windows Metafile output but it didn't work if I examined the file on a Mac. It looks like I am stuck with a raster image format, so I'm using PNG. Various FAQ lists I have found suggest using 1024×768 resolution.

R will produce its own titles so I don't want to mess with PowerPoint's slide templates. They also have very generous margins, but I want to use as much of the projected area as possible for my plots. Clearly I have to stop somewhere or I will be in danger of the edges being cut off, and since they have the slide title and axis labels, I'd rather avoid that.

So what's the minimum margin space I can use without having to worry that somewhere the stuff inside the margins will be clipped? A suggestion measured in pixels would be most helpful.

II previously asked this question on Microsoft's PowerPoint newsgroup, but no one felt able to give me a minimum number. I'm hoping that there will be some suggestions here with the wealth of PowerPoint experience at MetaFilter.

Please don't suggest using another software package instead of PowerPoint. I am well aware of the alternatives.

GLM Variable Weighting

Jimbob — Thu, 15 Mar 2007 21:24:04 -0800

Statistics. Generalized Linear Models; What's being done in this R script I've been handed down, with the variable weights? I've been handed an ancient and archaic R script that I'm using to do some variable selection, using generalized linear models. I'm trying to work out what it's doing, and if there's some name for the method of variable weighting it's using.

Essentially, it runs a GLM on a null model, a model saturated with all variables, a series of models that consist of the null model with each variable (ie. each variable on it's own), and a series of models that consist of the saturated model without each variable.

It calculates the % deviance explained for each of these models.

It then calculates the "change of deviance" for each model - in the case of the "null + variable" models, this is the additional deviance explained by the model with the variable, as compared to the null model. In the case of the "saturated - variable" models, it's the decrease in deviance explained of the model without the variable, compared to the saturated model.

In other words, it's basically working out how much of a difference each variable makes to the model; adding a variable to a null model might produce a great increase in deviance explained, but subtracting that variable from a saturated model might decrease the deviance explained only a little, because other variables still in the saturated model might still contain the information of the missing variable. This variable, therefore, has less explanatory power than a different variable that contains unique information.

After it's calculated all these "changes in deviances", it somehow converts all this into a weight for each variable (this is where the R-script loses me, and I can't figure out what it's doing). In any case, it gives output like this. This "+" and "-" signs indicate if it's reporting a null model + the variable, or the saturated model - the variable:

                        pcdev  		ch.dev

+DWMeanMin              21.0156 		21.0156

+DWMeanTemp             8.7369  		8.7369

+DW3pmTemp              3.5637  		3.5637

-DWMeanMin  		32.5202  		2.3141

-DWMeanTemp 		32.9797  		1.8546

-DWMeanMax  		33.0875  		1.7468

-DW3pmTemp  		33.8545  		0.9798

+DWMeanMax   		0.4372  		0.4372



Weights:

 DWMeanMin 	DWMeanTemp  	DW3pmTemp  	DWMeanMax 

    0.5739     	0.2606     	0.1118     	0.0537

Which is saying that variable "DWMeanMin" is really useful, containing a lot of information the other variables don't. "DWMeanTemp" might also be useful. "DW3pmTemp" is a long shot, and "DWMeanMax" doesn't contribute much explanatory power at all.

I understand the output. And it's useful. I can post excerpts of the R script if required. But what's this method of variable selection called?

How can I import high-quality (ie vector) PDFs into Microsoft Word for OS X?

docgonzo — Fri, 23 Feb 2007 12:36:57 -0800

Why does Microsoft Word (OS X v.11) hate PDFs? Or: Please help me make this stats assignment just a little less frustrating. One of the (many!) advantages of R is that it produces crisp and clean plots, exporting them as vector (ie PDF) files. Great.

However, when I add them to my DOC file (using "Insert > Picture > From file...") the resulting object in the text is a low resolution picture (possibly a rendered GIF or JPG.) Ugly (and, in complex plots, nearly unusable.)

How can I get Word to insert the proper high-quality PDF? I've hunted around the web and Word and found nothing (ie neither "Insert > File..." nor "Insert > Object..." results in success.) I suppose I could render the PDFs (ie through Photoshop) into something high-quality, but that seems excessive (and would be a pain); nor do I want to layout this multi-page assignment in InDesign (or some version of TeX!)

Thanks!

Statistical analysis package for OS X needed.

docgonzo — Fri, 17 Mar 2006 08:18:51 -0800

SPSS is fine but produces offensively ugly figures. R has nice, clean output but is otherwise inscrutable. Statistica is, alas, Windows-only. Mathematica isn't really it, either. What statistics package should I use for OS X? I am in the sciences and need to do basic statistical analysis up to and including regressions. Hopefully, I will be doing more and more statistical analysis -- if the gods of grad school align as I hope -- and need a package that is adequately open-ended to include newer techniques like Bayesian and MCMC.

What should I use?