Ask MetaFilter questions tagged with databases

How to beef up on basic Python/Java and get the skinny on databases?

foxy_hedgehog — Tue, 05 Mar 2013 19:02:42 -0800

I'm interested in taking this Coursera course on data science but the description states that "You will need basic programming experience with Java or Python, and some familiarity with databases." I do not have this. How could I get it, preferably online? I'd be grateful for recommendations for any sites or tools that offer exercises, tutorials, explanations et al. on the relevant languages and on the subject of databases.

I have zero experience in programming aside from some very simple exercises in BASIC decades ago. I have no idea how easy or difficult it is to require "basic programming experience" in either Java or Python, and I have no familiarity with the technical aspects of databases. If you think it's unrealistic to take this course, I would welcome your opinion and/or suggestions for alternatives.

Blind me with science.

deludingmyself — Mon, 10 Dec 2012 22:24:02 -0800

Help me discover recent 'Best Seller' scientific journal articles to read in 2013. Thanks to a graduate course I'm taking, I just rediscovered "The Hallmarks of Cancer," a fantastic review article published in the journal Cell in 2000. (That's right, it's a scientific article with its own Wikipedia page.) I read this in college in 2003, and it's one of very few papers that stuck with me from my undergrad days. As of 2011, it was also Cell's most-cited article, and so popular that the authors revisited it to put out a sequel to it the same year.

I want to read more papers like this across multiple disciplines. Not necessarily great research, but fantastic "state of the field" review, synthesis, and novel thinking on how all the different pieces come together, and what that might mean going forward. Note: I'm biased towards the life sciences, but interested in other fields, too.

I've seen this thread and a few others, but don't want particular article recommendations or journals to subscribe to, just a methodology for how to find things. I've got a knack for good search strategies when I'm looking for a particular topic or piece of information, but when I'm just seeking "papers that are amazing," I'm not sure how to go about it.

Obviously I could just turn to a publication like Annual Reviews, but I'd like suggestions on other ways to discover fantastic (and ideally fun to read, at least for a huge science nerd) scholarly articles online, regardless of their publication. Are there end of the year "top 10 lists" I'm not aware of? Great indexing sites? Other strategies you use to find good reads that aren't just within one specific sub-discipline? I'm vaguely aware that I might be able to use impact factors to hunt down most-cited papers, but not sure of the best way to go about that. Thoughts?

I'm a remote student, so assume I have excellent database access, but won't be walking into a library.

Access Concatenation Crazy!

horseblind — Wed, 25 Jul 2012 17:52:09 -0800

In Access 2010 I've come across a need to concatenate fields from one column in one table into a single, comma delmited value in another field in the same table (or in another table if I absolutely must). I've been working on an access database and have hit a wall after MANY hours of work and being THAT close to wrapping up my work.

I've read much on how todo this and it seems Allen Browne has it licked I've written the module as instructed and that comiles just fine. I've had many other different errors that I've gotten past and am down to getting a syntax error.

I have a table called tbl_phoneSources. The table is currently laid out like this:
Phone LastName Sources
__________________________
5552515392 Gates Source4
5552515392 Gates Source2
5552515392 Gates Source1
5552201356 Ellison Source3
5552201356 Ellison Source1
5652201356 Jobs Source4
5652201356 Jobs Source1

I want it to become:
Phone LastName Sources
_________________________________________
5552515392 Gates Source4, Source2, Source1
5552201356 Ellison Source3, Source1
5652201356 Jobs Source4,Source1

After creating the module I have finally come down to using this expression in a fourth AS field in my query. Despite seeing MANY examples illustrating very much the same thing I want to do I just can't get past my syntax errors.

Here's what seems to work for most who also have issues:
ConcatRelated("source","tbl_phoneSources","[phone] = #" & [phone] & "# AND [lastName] = '" & [lastName] & "'")

Did I make a typo? Do I just not understand this?

Free MySQL hosting for a college course?

jragon — Sun, 10 Jun 2012 17:28:47 -0800

I'll be teaching a class on "dynamic websites" to some designers. What's the best (and free) way to get them playing around with MySQL? If I were teaching one person about LAMP, I'd spend the time getting everything up and running on their Mac. But a) we'll be in a computer lab so I'm not sure what kind of permissions I'll have on the Macs. b) I'd rather the projects be available online.

What's the best free and hosted MySQL/PHP setup? Is there one? We wouldn't need a ton of bandwidth or space, but we'd really like a database.

(and I'm open to alternate ideas, like teaching to a flat file database or something)

best way to migrate back into database work

parrot_person — Fri, 08 Jun 2012 02:15:55 -0800

I'm thinking of getting back into IT. What would you as a hiring manager want to see on my resume that might counterbalance the time I've spent out of IT? What would be the best way to approach this? After college I spent a decade+ as a web developer. I've essentially been out of this world for several years now, only keeping my hand in it by doing some freelance work helping small businesses with their web presences (virtually no database work involved).

I've been working mostly as a massage therapist during this time, and foresee a time within the next few years when, for financial and physical reasons, I will want to transition back into IT.

The parts of web development that I liked the most were data modeling and writing queries (admin is interesting too but I never did as much). I am not really interested in brushing up my front end skills.

What would be the best way to get back into IT, specifically working with databases? If you are a hiring manager, what would you want to see on my resume? Should I target lower-level job roles and start from the beginning again? Should I go back to school or get a certificate of some kind? My BA is not tech-related.

Nutrition information databases?

jsturgill — Sat, 04 Feb 2012 10:39:35 -0800

Nutrition databases for processed food products? I'm aware of the USDA reference database.

Is there a similar repository (repositories?) of nutrition information for commercial, processed food products? Things like chain restaurant meals, fast food, frozen dinners, chips, etc.

I'm looking for sources of structured data, rather than websites like livestrong and whatnot.

Looking for resources about growing and managing an association's database

ladygypsy — Tue, 17 Jan 2012 09:19:02 -0800

As of Monday I'll be new to the field of Association Management. Can you share with me your favorite resources and best practices (I hate that phrase but it's the best I can come up with) on how to grow and wrangle a membership database? I'm switching gears in my work life - going from a newspaper where I've done online production-type work for 12 years to a trade association for a certain type of doctor. I'll be in the Membership department working closely with the database of members and non-members, and acting as a liason between the Database Admin in IT and the people in Marketing and Membership who need info from that database. I interviewed with many people and they all know of my lack-of-association-management background. The position is new, so I won't have the benefit of learning from my predecessor's notes.

I'm confident I can re-learn some of the database integrity practices I used here years ago and can learn to use their membership software (iMIS). But I'm looking for ideas, resources and case studies on how to grow the non-member portion of the database. Much of this will be on the job learning, but I would like some reading material to give me background so I don't start Monday as a completely blank slate. Google searches give me non-relevant associations to join, and opinions on how to grow email marketing opt-in lists. I found ASAE's Acronym blog shortly after I got the job and have it bookmarked.

So, Association Management AskMeFites, can you share with me your favorite resources for dealing with your database? I'll read blogs, read online articles, read books, join websites, follow Twitter hashtags -- you name it. Actually, any tips for thriving in this new-to-me field would be welcome since my new job jitters are quickly multiplying.

How do I create my own Wikipedia?

Pseudology — Sun, 25 Dec 2011 11:07:13 -0800

I'm volunteering at the Whistler Museum, it's just a little municipal museum, and our archives haven't been digitized yet. If it was possible to turn them into a Wikipedia-style database. Example inside. Example:

I'm doing an article about the logging camp/ghost town of Parkhurst and I was hoping it would read something like this.

"Parkhurst was a lumber mill and community on Green Lake(link to article on Green Lake) set-up and run by Ross Barr(link to article on Ross Barr)."

Is there a program that could let me write that, either for a website or just for a private database?

It would also be nice to be able to scan documents and tag them to their relevant subjects (and link to them in citations) as well.

What's the best CMS for this project on banned books

johnofjack — Mon, 12 Sep 2011 15:28:47 -0800

What's the best CMS for this project? I wanted to put together a database of books which were actually censored somewhere somehow in the U.S., preferably recently.

Drupal doesn't appeal to me anymore --I have a fair amount of experience with it but am still frequently stumped about why it is or is not doing one thing or another, and I'm consistently unimpressed with how slow it is. So I'd be surprised if Drupal is the answer to this question.

Info to store (at minimum): title, author, cover, plot summary, intended audience, dates censored, locations censored, type of censorship (e.g. removed or restricted), external references, and keywords.

The dates, locations, types of censorship, and external references should all be grouped as individual instances of censorship tied to a given book. (If you know anything about databases, I'm seeing books as a table, authors as a table, and incidents of censorship as a table, with links between incidents and books.) Authors and books would have their own interlinked pages, as would audience and types of censorship.

I'd thought of just using WordPress for this, but where I'm stumbling a bit is in making it automatically update various pages like authors and books. For instance, on a book's page I'd like it to list incidents, keep an automatic account, and possibly map locations. I'm not sure of how these things might be done in WordPress.

Is there anything set up that's not Drupal that can likely do this (even if it's WordPress and a number of plugins), or am I stuck with Drupal?

In case you're curious, the project is in response to the ALA's Banned Books Week, prompted by my belief that challenged is not the same as banned any more than bruised is the same as dead.

Geo Database

digdan — Fri, 02 Sep 2011 15:21:16 -0800

Where I can find free United States geographic databases? I need a database with relationships between all zipcodes, areacode,cities,states,counties,neighborhoods, all with coordinates and detailed information if possible. How up to date does not matter, and I am looking for something free.

Seeking advice on flickr maintenance and interface.

mortaddams — Thu, 21 Jul 2011 19:01:33 -0800

Seeking advice on good tools for automating the maintenance of a professional flickr site, and ways to make an interface to it. I am in the process of making thousands of professional-quality photos available through flickr, along with some friends. We love the idea of using flickr to host our images because of the size of its user base, and we do not want to fully reinvent the wheel for sharing photos. We have already uploaded and tagged half of our images.

Now I am wondering if there are any good programs to automate the maintenance of our flickr site. I would be especially interested in programs that allow us to do things like change all of the descriptions of photos in a set at the same time, or add all of the photos tagged in a certain way to a set.

Also, are there any ways to use flickr as a back-end for a photo database, such that we could use our own non-flickr website as an interface to our images on flickr? Unfortunately our web-development skills are limited, so we would only be willing to try easy-to-implement solutions.

Thanks in advance for any advice that you can provide.

Please help me research consumer demographics.

brynnwood — Wed, 01 Jun 2011 15:06:41 -0800

I'm looking to research consumer demographics for a couple of specific businesses. For instance, Who goes to Disney World or Cirque du Soleil (age, race, income, city/state of origin, etc.)? I realize this is pretty basic stuff, but it is not anything I've done before and I'm coming up empty. I'm thinking theme park or live entertainment attendees (and their demographics) have been studied and a report created for "public" consumption via a marketing database or other. I've tried redbooks and mintel, but I'm really new at this sort of thing. (Also, redbooks subcription is too expensive at the moment.) So other than those and google, I'm at a loss. I'd really appreciate hearing from any marketing/advertising experts on this. My new job sort of depends on figuring this out.

I'm working on strategy for an advertising client and wanting to understand these competitors' audiences first.

Thanks so much for your help.

Best certifications and online courses for a new database administrator?

anonnymoose — Wed, 23 Feb 2011 15:04:39 -0800

Best certifications and online courses for a new database administrator? I know this was asked back in 2008, but I figured it was so long ago that it might be worth asking for more current opinions. I love databases. I use them at work all day, every day, for high-level searching, and now I want to try administering them. Where to start? What are the best certifications to get my foot in the door with an internship? Are thee any online classes you'd recommend?
My local community college is pretty much a bust. I'm considering the O'Reilly Technology School database cert or Oracle certifications, but are either of these worth the time/money?

And, further on, where to go from there to gain some experience? Does volunteering myself as free labor to get se experience (the good old-fashioned way) apply here? If it matters, I'm not a student, and I work full-time at present. I would like for this to be a career transition, if the fit is good. Any help would really be appreciated.

Can you make my brain my screenshot?

fritillary — Fri, 18 Feb 2011 18:58:14 -0800

I'm diagramming the history of visual arts in my city. I need a "database" -- I use the term loosely -- to help me keep the connections straight. Does such a thing exist? Think Facebook meets social networking visualization meets Internet Archive. (The Thing, the Meta-Thing, and the Evolution of Both!) I need to be able to visually situate thousands of names, places, organizations, artworks, and personal connections. I'm covering roughly 150 years, but most of the research is just evolutionary footnote... my real interest is the current state of affairs. So I'd also like to add photos and link to blogs and homepages of active artworld figures.

I realize this question is quite broad, and I'm probably best accommodated by a combination of memory aids. But my project is still in infancy. Right now I've hopping like a madwoman between endless lists of "who's who"... matching snapshots and art clips... a running current events calendar... stacks of library books and historical errata... my personal notebooks... and pages and pages of web-links. There must be a fuzzy solution to this linear problem!

(I should note, I don't need to store huge text or media files... I'm not interested in the "meat" of data, so much as the social organization of cows. Webs of Proper Nouns is where it's at yo!)

OpenSourceMusicGeekProjectFilter question

LMGM — Wed, 02 Feb 2011 20:56:47 -0800

I need some advice on how to build a database-driven site around an archive of ephemera (mostly flyers). Essentially, I want to build an online archive of scanned rave flyers from the Toronto-London-Detroit rave scenes (plus some Midwest US flyers). Here's what I have: Here are my resources so far:
- a dusty file folder full of about 10 lbs worth of flyers, pamphlets, and mini-zines from the rave era from 1995 to the early 2000s.
- through a university library, access to a scanner where I can get full-color images
- an undergraduate minor in comp sci, which gave me some basic project-development workflows plus an intimate knowledge of C++, Java, and Perl.
- Some experience coding HTML and CSS, particularly through Dreamweaver (which also does PHP)
- Some self-taught proficiency with MySQL and PHP
- An installation of MySQL on my home computer for testing
- space on a server with MySQL and PHP activated.

So, I want to be able to scan a pile of these flyers, attach metadata of various sorts to them (location, date, headliners, description of event, etc), and present them as a searchable, web-accessible archive. Any advice? re: platforms, data structures, useful utilities and tutorials, examples to work from, etc…

I'm a bit of a bedroom coder and actually get a kick out of coding things by hand from the ground up, so I'm not looking for solutions that involve plugging one online app into another and putting a glossy skin over it.

Seeking ANG Newspaper Group fulltext archives

brainwane — Tue, 11 Jan 2011 06:47:23 -0800

I want to get the full text of several articles I wrote that were published in Bay Area newspapers between 2005 and 2007. What periodicals database should I be hitting? Preferred sources: free or accessible via New York Public Library. Between 2005 and 2007, I wrote several articles for the ANG Newspaper Group (Oakland Tribune, Tri-Valley Herald, and several other papers) in my weekly column "MC Masala" (byline: Sumana Harihareswara). I'd like electronic copies of the pieces as they appeared in print and online at insidebayarea.com. I have my drafts but I'd prefer having copies of the edited versions.

I estimate that I put out about a hundred pieces. Archive.org doesn't have them. The insidebayarea site only lists 61, and would charge me more than a hundred dollars via newsbank.com to get access to them all. I'd prefer if I could use an electronic database that I already have access to via my NYPL membership, especially since the insidebayarea/newsbank search is missing many of my articles anyway.

I've looked at the periodicals databases that NYPL lets me access and tried EBSCOhost and GALE's Infotrac; they either don't have any ANG archives or the archives aren't deep enough. The next time I physically visit the library I intend on trying ProQuest. I remember Lexis-Nexis being a go-to source for this sort of thing, but NYPL does not seem to provide it.

So: what periodicals database should I be hitting to get the full text of about a hundred feature articles published in ANG newspapers between 2005 and 2007? I would also welcome advice that tells me "contact such-and-so at the Oakland Tribune to get softcopies of everything"; their contact page doesn't list an archivist so I'm unsure whether they still have such a position.

(And I have a question in to the NYPL staff about this, too; if they respond, I'll post the answer here & close the question, but I haven't had much luck trying to get this kind of information/help from the library before.)

How to delet things from google post death

marimeko — Fri, 17 Sep 2010 20:25:28 -0800

How to delete things from google post death? I found out today that my Dad's business (name of business, related phone numbers, address, etc) are still all over google as though his business were still up and running. This was the first time I googled his name since he died nearly five years ago. I find it (at minimum) misleading and (maximum) jarring/horrifying (particularly when I think of those people who may not know he is dead).

How do I permanently remove my Dad's business info from google?

How do I NoSQL?

Deathalicious — Sat, 10 Jul 2010 11:42:02 -0800

My bread and butter is and always has been data stored in relational databases. I keep seeing a lot of material on key-value storage (also known as the NoSQL movement) but haven't had as much success finding resources on how to move from relational to non-relational. Any suggestions? I know that some of this involves denormalization, but how? Step-by-step guides where someone takes a heavily relational database and moves it to a key-value system would be appreciated, as well as gentle guides for the non-relationally challenged.

The (Facet-Searchable) Paperless Office?

Alterscape — Fri, 04 Jun 2010 07:05:49 -0800

Are there any commercial products intended to (or could be set up to) mine content from unstructured office documents (think: memos, presentations) and populate fields in a database for faceted search, or is this still the domain of research? I'm looking for software that, given a vague idea of the structure that the documents probably have, does Named Entity Retrieval / matching against named entities in a known list, classifies documents based on training data, extracts chunks of text that match certain patterns and inserts them into a database as fields for future faceted search, that kind of thing. The documents need to remain editable (so that, eg, sections of a PowerPoint can be copied for re-use once located).

I know a lot of full-text indexing/retrieval (IMR Alchemy, etc) software, but most things in that space are archival (eg, doesn't work with editable documents), and as far as I can tell most can't do automated indexing for faceted search.

I work with folks who do research in areas surrounding IR and document analysis, but I'm not aware of any off-the-shelf products doing this kind of work. I suspect it's kind of a holy grail in IR and if anything exists, it's in the lab and not entirely stable. Are there any commercial products doing the build-a-faceted-index-based-on-content-analysis stuff, though? Assume cost is no object.

Using Filemaker for a Not-So-Pro

nkknkk — Wed, 19 May 2010 12:16:39 -0800

My dad has inherited an old Filemaker Pro 7 (Mac) database. He needs to access and update the data... from his Windows machine, without Filemaker. Help! My dad isn't a luddite, but isn't particularly tech-savvy ("How do I use the Wikipedia?"), and I know nothing about databases at all. He's been put in charge of maintaining a club history - names, addresses, activities etc of members, that stretches back at least a decade, and has been stored in Filemaker Pro (Mac). The most important use of this information is to print a weekly roster of participants, ranked in a certain way.

How can he get to the info from this old Mac file into something he can manipulate with Windows? Does he need to buy Filemaker for Windows (and will that work)? What's an easy (easier!) way for him to manage and maintain this information over time?

Help me create a bad-ass program for managing TB data

McBearclaw — Sun, 21 Feb 2010 21:49:05 -0800

I'd like to take on a somewhat substantial software project that should a) be open-source and b) not suck. Suggestions on how to code for others requested. I work for a tuberculosis clinic. TB is a magnificently data-intensive disease, but the software we have to manage those data is not as good as it could be - although, despite being written by one heroic programmer, it is basically as good as any commercial demos I've seen. The problem is that none of these programs really address the workflow of a TB clinic - they tend to combine both data entry and data display into a form which corresponds to exactly one table. A few customized forms would really cut down on the time my colleagues spend on this stuff. Did I mention that everybody can edit everything?

I have magnificent visions of an open-source program for TB clinics. I'm essentially a programming novice, but I think that, with learning, the actual coding of much of this program would be within my grasp - the bulk of it would just be querying, displaying, and editing text. As a data analyst, I have a solid grasp on what I would want from such a system, and I have ready access to the opinions of the clinical workers. I'm not sure I really believe that I can make a viable product, but this is something that I really want to do, from which I would learn a lot, and that might ultimately make something good.

What I don't have is experience writing code that is digestible and extensible for others. I'd like for other clinics to be able to take this code and customize it to their computing ecology - maybe they need different forms, or need it to link to a different appointment scheduling or laboratory system.

My most extensive experience with a programming language is with R, which won't be of much help here. I am comfortable with SQL, and write most of my (small, simple) utilities in Python. I'm open to learning any language/framework that will help me get this done right.

Your input on any aspect of a project like this is greatly appreciated (even if it's just to say, "You're way out of your league, and here's why.") I will try to promptly answer any questions.

AdoDB Gurus: Relate me!

gaby — Thu, 11 Feb 2010 08:46:02 -0800

ADODB Gurus: Many to many relation, how can I simulate this using AdoDB for PHP? I'm loving the ADODB_Active_Record class, really helpful. The problem is that I have a relation that looks like it requires an intermediary table to link the data up. The relation is:

- A school can have many employees
- An employee can only work at one school
- Employees have a user account on the system, as well as administrators, teachers, etc.

I have a users table that holds the user accounts (employees each have a user account), I have a schools table and I have a users_schools table which links user_id to school_id. Using this intermediate table and some plain SQL I can list who works for a given school. Adding a school_id field to the users table seems the wrong approach, that's not what the users table is for so I need the intermediary table to link users to schools.

The issue is that I'm using AdoDB and Active Records so I want to either stick to using plain old SQL or Active Record. There are some one to many functions built into AdoDB but I can't see one that will let me use the intermediary table directly.

There must be a simple way round this, the problem itself is as old as the moon and must have been solved lots of times. Googling for the problem returns lots of ADO.Net stuff which is not helpful. Can you help an old tech out?

MySQL and PHP suggestions for beginner?

parrot_person — Mon, 18 Jan 2010 05:52:34 -0800

I am looking for suggestions for inexpensive online or real life courses, books, or web sites that will help a friend learn MySQL and PHP. Hello,

I have a friend in Dudley, UK who would like to get ino database development, MySQL in particular. He also would like to know at least the essentials of a scripting language such as PHP in order to be able to create web sites that can interact with any databases he creates.

Thus far he has made progress learning data modeling concepts, but is a bit stuck in implementing them by actually building a database in MySQL (though he does seem to now have a working installation of MySQL on Fedora).

He recently took the "Introduction to Database Design" course from eclasses.org. Although he definitely learned from the experience, a class with more feedback would be preferable. I was able to help him some because I am a web developer and pretty solid on data design concepts. But I will be able to help him less as he gets into MySQL because I have almost no experience with that db.

I know there are free resources on the web, but since neither of us is that familiar with MySQL, we don't know which resources are best. And we also think that taking a course or following a lesson plan will help him more initially by giving him a specific path to follow, rather than trying to wade through the tremendous amounts of info online.

We are looking for suggestions for inexpensive online or real life courses, books, or web sites that will help him learn MySQL and then PHP. And more generally, any suggestions you might have for what to focus on, what to watch out for, etc. I also think it might help for him to be able to ask the occasional question of someone who is truly knowledgable in MySQL and/or PHP.

These are the books he has in his possession thus far: Databases Demystified by Andy Opel, PHP6 & MySQL by Larry Ullman, and MySQL pocket reference by George Reese.

Thank you so much for any suggestions!

Learning databases through selecting stars

StephenF — Wed, 06 Jan 2010 09:50:44 -0800

Help me find a tool which helps you learn how to use databases through selecting stars. I seem to remember seeing a tool about 18 month ago which helped you learn how to use databases through selecting stars - you'd run a query on particular star types, and it would show you which ones you had selected. I think it was a downloadable application for Windows, and there were various exercises you could complete.

Does this ring a bell for anyone? If not, alternative solutions for learning MySQL which allow you to run queries and see the results graphically much appreciated.

Insert data into several tables via form (MS Access)?

speeb — Tue, 05 Jan 2010 19:17:30 -0800

I'm working on an Access database that collects various metrics and numbers weekly. I've used week numbers as the primary key in each table since that's the only thing that can reliably group them together. If I were to build a form to input data into more than one table at once, is there a way to enter the week number only once on the form, but have it insert the number into multiple tables? Or am I doing this wrong altogether? This database basically collects information about performance for 6 different departments that all work together and share some management. The whole point is to look at this information on a weekly basis, hence my use of week numbers. I'm very new to Access and could probably do a lot of things differently, so I'm more than open to suggestions.

I currently have at least 2 tables for each department (one for phones, the other for service) and a table of just weeks (week #, start date, end date).