Data, R

R and Access – for loops

In this post I demonstrate how I use Access together with R to help dealing with unexpected crashes and other mishaps that might occur when running R on a weak and old machine like mine.

Features: RODBC package.

You can check my earlier post on R and Access if you want to set that up for yourself.

As we often do on SocialFunction(), we’ll get some tweets using a couple of keywords. We could just:

search_terms <- c("social", "media", "stats", "R", "#rstats", "r-bloggers","Microsoft Access",
                  "twitter", "R package", "etc")

tw_df <- data.frame()
for (i in 1:length(search_terms)) {
  tweets <- searchTwitter(search_terms[i], n=2000/(length(search_terms)), lang="en")
  tw_df <- rbind(tw_df, twListToDF(tweets))

However if your machine crashes half way through (i = 5) the output of the job is entirely lost since tw_df isn’t really saved; R objects are kept in the physical memory (RAM).

I don’t have a very good pc or a good internet connection; crashes, freezes and other misfotunes that break for() or while() loops sometimes happen. It is incredibly frustrating when a time consuming loop breaks and you have nothing to show for it.

There might be more elegant ways to deal with this but I simply use an Access table like you would use an R data.frame() object – append the results of every loop in an Access table (instead of a data.frame) and the results will be stored outside of your R session and on your actual hard-drive, these are saved no matter what your machine throws at you.

search_terms <- c("social", "media", "stats", "R", "#rstats", "r-bloggers","Microsoft Access",
                  "twitter", "R package", "etc")

con <- odbcDriverConnect("Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=D:/database.accdb")

tw_df <- data.frame()
for (i in 1:length(search_terms)) {
  tweets <- twListToDF(searchTwitter(search_terms[i], n=2000/(length(search_terms)), lang="en"))
  sqlSave(con, tweets, tablename="example", append=TRUE, rownames=TRUE)

In hope this helps some of you.




  1. Pingback: Twitter crawler with R & [DBM] | SocialFunction() - December 17, 2014


Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: