How to fix serialization and stream corruption errors in Java program?

May 15, 2015 8:01 AM

I'm pretty new to programming and I'm trying to resolve an issue with a java-based program I'm using for a research project. I've downloaded/installed the program files and updated the directories, but when I run the program I get an "InvalidClassException" error. The bigger problem is that when I try to define my "serialversionUID" I get another error: "StreamCorruptedException". I'm running the program in my terminal on my Mac. I'm not sure what I'm doing wrong or how to get the program to run. Any help would be greatly appreciated.

The program I'm trying to run is "Personality Recognizer" written by François Mairesse. The program files are available for download at: http://farm2.user.srcf.net/research/personality/recognizer.html

The serialization I get when running the program initially is:

java.io.InvalidClassException: weka.classifiers.Classifier; local class incompatible: stream classdesc serialVersionUID = 7126777525038658319, local class serialVersionUID = 6502780192411755341

I've tried to apply a solution found on stackoverflow to resolve the serialization error (solution at: http://stackoverflow.com/questions/13599217/how-to-get-rid-of-invalidclassexception-serialversionuid). Following the solution in stackoverflow, I added the following code to the file where the problem is taking place:

private static final long serialVersionUID = 7126777525038658319;

After doing so it gives the following error:

java.io.StreamCorruptedException: invalid stream header: 70726976

Stackoverflow gives also gives a solution for the corruption error (http://stackoverflow.com/questions/23262160/java-io-streamcorruptedexception-invalid-stream-header-54657374), but I can't make sense of it in my case.

Please let me know if I'm more information is needed. Thanks in advance for the help!

posted by jharrison to Computers & Internet (5 answers total)

Can you tell us what steps to reproduce the problem? It doesn't look like the dictionary is available to download, but im not sure how required it is.

Did you try recompiling the personality recognizer source? or are you using the prebuilt jar/.class files?
posted by TheAdamist at 8:36 AM on May 15, 2015

Best answer: This is a serialization problem; in Java, serialization is the in-built mechanism by which objects are written to disk (or other media). All serializable Java objects (that is, objects which implement the marker interface java.io.Serializable) are assigned a version number by the compiler or the programmer (that is, the former does it if the latter doesn't explicitly define a value by setting that special private static final long serialVersionUID). If the compiler generates the value, it's based on properties of the object itself, and therefore changes when the source for the object is changed and recompiled (e.g., its fields are changed, usually by adding or removing). What seems to be occurring is that the Classifier instance being deserialized does not match the version compiled by you, probably because the object's definition has changed. weka.classifiers.Classifier is itself an interface (that doesn't implement Serializable) so I'm wondering where you introduced the 6502... value -- did you leave off the part of the error output where it names the actual class that won't deserialize properly?

Assuming you foudn the right object, introducing the 6502... serialVersionUID into the source may not fix the problem because the version really has changed (that is, the default deserialization mechanism doesn't have the old object anymore, so can't properly derive the format of the serialized object). I suspect it's something in the lib/models directory that contains the offending data.

What happens if you use the binary downloaded from that page rather than compiling your own version? Have you modified the source or attempted to use a different version of the Weka library than 3.4.3?
posted by axiom at 8:51 PM on May 15, 2015

I sent you memail - i might be able to help off-line.
posted by nightwood at 11:07 AM on May 16, 2015

Response by poster: Hey folks, thanks for the comments and sorry for the delayed reply. I went out of town after posting this and thought I'd have access to internet while I was away, but that didn't pan out. Some replies to your questions:

@TheAdamist: I think the full dictionary database can be purchased for download at http://www.liwc.net/, but the required file is also included in the author's source files (lib/LIWC.CAT). I just used that file.

I'm using the prebuilt files provided on the Personality Recognizer website. Really all I've done is download the source and binary files and followed the instructions under the "installation and usage instructions" on the website (http://farm2.user.srcf.net/research/personality/recognizer.html), using the Unix specifications. After that, I tested the program by running it on my terminal to try to duplicate the example in the output.txt file. So I used the following options: ./PersonalityRecognizer -i examples -d -t 2 -m 4

The program begins to run properly (i.e., it loads the LIWC dictionary and MRC database), but as Axiom suggests, the error occurs when loading the model (lib/models/self/SVM/std-extra.model).

@Axiom: The only things I've modified are the PersonalityRecognizer.properties file and the PersonalityRecognizer shell script, according to the instructions on the website. So as far as I'm aware, I'm still using the binary downloaded from the page. Or am I missing something? I also have not used another version of the Weka library.

I did leave off the part of the error output with the class that won't deserialize, because there are actually several. Here is the full output:

java.io.InvalidClassException: weka.classifiers.Classifier; local class incompatible: stream classdesc serialVersionUID = 7126777525038658319, local class serialVersionUID = 6502780192411755341
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at recognizer.PersonalityRecognizer.loadWekaModel(PersonalityRecognizer.java:791)
at recognizer.PersonalityRecognizer.loadWekaModels(PersonalityRecognizer.java:567)
at recognizer.PersonalityRecognizer.loadWekaModels(PersonalityRecognizer.java:532)
at recognizer.PersonalityRecognizer.main(PersonalityRecognizer.java:295)

But from what you say, it seems like I did not introduce the SUID properly, anyway. I tried to apply it at the top of the std-extra.model file, but do I need to introduce it for every location listed in the error?
posted by jharrison at 10:20 AM on May 21, 2015

Response by poster: Correction: I actually was using a newer version of the Weka library. I forgot I had installed the most recent version (3.6.12) from the Weka page (http://www.cs.waikato.ac.nz/ml/weka/downloading.html) thinking it would provide the most up-to-date results.

After removing that version and installing 3.4.3, the program now runs properly. Thanks for pointing me in that direction, Axiom!
posted by jharrison at 10:41 AM on May 21, 2015