Comments on: Migrating vBulletin to Newsgroups?

Question: Migrating vBulletin to Newsgroups?

pazazygeek — Tue, 14 Mar 2006 11:14:12 -0800

Best way to export posts from a vBulletin forum and repost them to newsgroups? I'm not a programmer and am not going to be doing the work for this, but I'm trying to find any plugins or tips to help the person who's tasked with this. We have a vBulletin forum with approximately 35 boards on it, and each of those boards has a pretty decent number of threads on 'em.

I'm looking for an easy way to get all of the relevant data exported from the mysql database to text so that all of those posts can be migrated to a newsgroup, as quickly as possible. There are lots of tables with irrelevant info in the db.

We don't care about attachments, we do care about formatting. Anybody have experience with this sort of thing, or know of any good resources, plugins or existing scripts? I've found a plugin for vBulletin called ImpExp, but that's designed specifically to migrate from one vBulletin to another, so isn't particularly helpful.

By: orthogonality

orthogonality — Tue, 14 Mar 2006 11:39:25 -0800

Assuming the person who is tasked with this has some knowledge of SQL and can examine the data in the database, this should be a trivial if tedious project. If he can't do it, it's because his knowledge of SQL or general problem-solving skills are unequal to the task, and you need to get someone for whom it is trivial.

The number of boards and the number of threads are not a real factor in whether this is difficult.

By: pazazygeek

pazazygeek — Tue, 14 Mar 2006 11:42:40 -0800

Yeah, he can do it. It's just a completely tedious and time consuming thing that I have a feeling might have already been solved somewhere out there on the internet, but my searches thus far have been fruitless.

By: empath

empath — Tue, 14 Mar 2006 12:01:12 -0800

If I were doing it, I'd modify the external.php code to dump the entire contents of the messageboard as a gigantic xml file.

I'm not good enough with php to feel comfortable telling you how to do that, though, but I'm sure it wouldn't take much time to noodle out if you're at all good with it.

By: empath

empath — Tue, 14 Mar 2006 12:10:02 -0800

To elaborate a bit:

external php is a script built into vbulletin that is usually used to create rss or xml feeds of recent threads.

For example:

http://www.forumurl.com/backend.php?type=xml&forumids=1,2,3,4

would give you an xml file of the 15 most recent posts(total) out of those 4 forums, but only the title and a link to the threads.

I believe that the only thing you would have to modify is to remove the limit of 15 posts (it's got a hard limit of 15 coded in, I believe) and to add in whichever additional fields you need to add. I'd suggest for obvious reasons to copy the external.php file to a new name before modifying it and delete it when you're finished with it, because you wouldn't want anybody else to stumble onto that functionality.

By: kableh

kableh — Tue, 14 Mar 2006 14:01:20 -0800

Well something's hosed with my board now, but have you looked at the ImpEx addon for vB? I know it can import many different formats, but I can't get into it at the moment to check what export functionality it has.

You'll need your customer number/password to login to the members section of vbulletin.com to download the latest version.

By: Rhomboid

Rhomboid — Tue, 14 Mar 2006 18:16:58 -0800

kableh, read the question again. :-)

If it were me I would forget about doing this through the board and just write a perl script that reads each post in the database and then uses something like News::NNTPClient to post each message to the newsgroup server. Sure you can mess around with RSS feeds and export plugins but I think this will be quick and direct if you have somebody with programming skills.

The only thing to watch out for is that if you want to maintain threading you'll need to do some translation of message-IDs. This should be trivial though because every message in the DB should already have a unique ID already, so just use that (or trivially modify it) as the Message-ID: for each post. And then make sure to include a References: or In-Reply-To: header for each post in a thread so that a threaded newsreader will be able to recreate the thread structure.

Regarding formatting, remember that NNTP is just plain text. So you will probably spend some time figuring that out too. If it were me I'd look into using HTML::BBCode on each post and then render that HTML to plain text that is wrapped at 80 columns (with something like lynx --dump or the equivalent perl module.) You could also do this directly with BBCode::Parser but that could potentially be more work.