Log in

No account? Create an account
LiveJournal Client Discussions [entries|archive|friends|userinfo]
LiveJournal Client Discussions

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Client feature request [May. 5th, 2004|02:37 pm]
LiveJournal Client Discussions

I'm currently developing LJBook. In order to avoid the login/password issue, I'm looking for developers of LJ clients which could implement a XML exporter for experiments with users (no need to export comments at this time).
So, it's "just": backuping a journal and outputing a XML file.
The XML would be uploaded to the web site and used for producing a PDF.

I made a DTD which is here: http://www.ljbook.com/blogbooker.dtd
or it's possible to use the same syntax as the XML outputed by jbackup.

The XML should be prevented from any user manipulation. (For example, during the development process, some users provided me files made from a cut/paste of a XML file outputed by IE or XML files were put in a Word document... etc...)

Thanks for any help!

[User Picture]From: hythloday
2004-05-05 08:56 am (UTC)
Assuming you mean something like this style, I'm in the middle of writing one, although it won't conform to your DTD - I'll mail you when I'm done, if you like.

It's reasonably easy to write your own, if you'd rather go down that route.
(Reply) (Thread)
From: gadsounet
2004-05-05 12:35 pm (UTC)
Do you plan to implement a GUI ?
(Reply) (Parent) (Thread)
[User Picture]From: hythloday
2004-05-05 05:31 pm (UTC)
Nope. It's for something slightly different.
(Reply) (Parent) (Thread)
[User Picture]From: premshree
2004-05-05 12:20 pm (UTC)
LJ Backup. It backs up your journal as a RSS feed.
(Reply) (Thread)
From: gadsounet
2004-05-05 12:35 pm (UTC)
I know your tool. I have seen it on lj_nifty. Despite it's very useful, it will not be for me.
I need some client with a GUI to implement it. It must be very easy to use. (I don't say running perl scripts is difficult, but it might be hard for newbies).

Thanks for your reply!
(Reply) (Parent) (Thread)
[User Picture]From: dneighbo
2004-05-05 01:14 pm (UTC)
Having used LJBook and being very impressed. What I would love to see is LJBook be a self contained program. This way it could be wrapped up as a plugin for a number of clients. It would be great if in my client, I could make a book right from its user interface.
(Reply) (Thread)
From: gadsounet
2004-05-05 01:29 pm (UTC)
I have thought about it before continuing to make it run on a server.
I don't think it will be easy to push this as a plugin.

It's because it uses LaTeX. LaTeX is great, but it's also a nightmare... and the distribution for Windows is at least 25Mb (just the installer). See here: http://www.miktex.org/setup.html

The dependencies for creating a LJBook for many users would bring it to at least 150Mb. It's like bringing the whole professionnal printer in your house.

And I don't think you would need to create a PDF more than 5-6 times per year...

But a lower quality book, could be done with some PDF libs. But I don't think it will ever allow to typeset like LaTeX/TeX.
(Reply) (Parent) (Thread)
[User Picture]From: dneighbo
2004-05-05 01:43 pm (UTC)
Could you make the server interactive with the client. This way the client need not dump an xml file for the user to manually upload and you could still make a plugin wrapper. Basically you could let the plugin implementor choose to either spit out a file and do the transfer and retreival via the client or you could use some sort of RPC to handle the entire transaction.

My point being, web applications suck. It would be nice to have all this functionality within my existing client even if my client is required to be connected to a backend for processing.
(Reply) (Parent) (Thread)
From: gadsounet
2004-05-05 03:05 pm (UTC)
Of course it would be possible to do it (and it would be great). The client would have then to "ping" the server, waiting for the PDF to be done...

But currently I'm looking for the basic stuff to work: a XML file -> PDF.

And it would require LJBook to live more than 4 months... (see gads_ljbook)
(Reply) (Parent) (Thread)
[User Picture]From: fg
2004-05-05 10:20 pm (UTC)
i've thrown together a little .NET app that downloads a journal and stores it as xml, and allows you to search, run plugins, and so on.

here's a screenshot of the archive client.

right now, one of the plugins outputs the whole journal to html, and from there you can print to pdf, but i'm not satisfied with the process. using IE to print to PDF is slow, takes up monstrous amounts of memory, and probably won't work on a standard user's machine.

here's a screenshot of the print preview.

i'm not sure if this is what you are looking for, but i'd be happy to work with you if you are. what i could do for you would probably be a plugin for my app that talks to your server.
(Reply) (Thread)
From: gadsounet
2004-05-06 05:51 am (UTC)
It looks really user-friendly!!! :) Great job!

Well, the PDF produced by IE or Mozilla will certainly be "good"... but LaTeX can do something even better I think (despite the fact that TABLES are not supported at this time).

That's what I'm looking for, yes!
Concerning the plugin, it would be the next step.

Could you generate a XML file people would upload to the web site ?
The blogbooker.dtd is just an improvement of the implicit DTD of export.bml.
(Reply) (Parent) (Thread)
[User Picture]From: fg
2004-05-06 08:44 pm (UTC)
the format in your DTD is pretty similar to how i already store the xml. it wouldn't take much work for me to implement.

i'm going to have time to do more work on it this weekend - my plan is to release it soon as open source. i'll drop a comment in your journal when it's ready and we can discuss this further.
(Reply) (Parent) (Thread)
From: gadsounet
2004-05-07 04:06 pm (UTC)
The DTD is just a little improvement from the implicit one generated by export.bml. Happy to see, it will not be a problem for you! :)

Tomorrow, I will setup a HTML page with a form. It won't support comments (do you backup comments ?!) nor include images. But it will be just a start... The rest will come shortly.

I will post the URL here. (People exporting with export.bml should be able to use it.)

If it works fine, it would be great to see how this could be nicely interfaced with your client.
(Reply) (Parent) (Thread)
From: gadsounet
2004-05-08 09:16 am (UTC)
Here is the URL to send the XML file to:


It will parse the XML according to blogbooker.dtd. It doesn't care about the root node.

I have tested it with some XML files generated by export.bml.

Let me know if it works or not.
(Reply) (Parent) (Thread)
[User Picture]From: fg
2004-05-08 04:26 pm (UTC)
i tried sending an xml file that followed your DTD. when i included the AUTHOR element it failed, but without the AUTHOR element it works fine.

you have an element in your DTD called CATEGORY - what is that for?

some lj users are going to have very large xml files. for example, my journal produces an xml file that is 1.12 MB, and i know people with MUCH bigger journals than mine. the reason i mention it is that sometimes file upload over http (like on your LJBook form) doesn't work well with large files. something to keep in mind!

oh, and to answer your question - my client doesn't download comments. right now it's not supported by the server protocol. i think there are clients that download comments, but if so they are probably doing it via screen scraping. personally i think screen scraping is evil     :)

if lj decides to support comment downloads, i'll integrate it into the client.

i need to clean up a few things still, but it's likely that i'll post the program up tomorrow so you can try it yourself.
(Reply) (Parent) (Thread)
From: gadsounet
2004-05-08 05:11 pm (UTC)
AUTHOR is related to comments, so it won't work.. I should add it to the root entrie...

Category is related to other blog systems like MovableType or WordPress where people can post an entrie related to a categorie like (Life, Computer, Music, Friends...).

Currently LJBook has already handled 12Mo files resulting in 3000+ pages PDF (yes, some people have HUGE journals! :) ). The issue you point out (I think it's this one) is related to some settings of Apache & PHP. Currently the limit is fixed to 8Mo, but I can increase it.

There is a way to download comments properly, it's here:
http://www.livejournal.com/developer/exporting.bml , I got a client running on LJBook created by marksmith (who impleted it on livejournal) which handles it quiet nicely. BUT, the jitemid is not related to the itemid you got from export.bml. Unless you use XML-RPC it will not work.

I will add support for Images tomorrow and create a new page ( http://www.ljbook.com/xml.html ) with a link to your client.
(Reply) (Parent) (Thread)
[User Picture]From: fg
2004-05-09 10:12 pm (UTC)
here it is.

it's still under development, but feel free to mess around with it. i tested the XML export with your LJBook and it worked fine.

you'll need Windows and .NET.
(Reply) (Parent) (Thread)
[User Picture]From: xial
2004-05-23 09:03 am (UTC)
Not sure if/where you would like bug reports with this.
Ran it this morning, got the following error:

ljArchive Error: Error logging in: CookComputing.XmlRpc.XmlRpcTypeMismatchException: response contains struct value with missing non-optional member: fastserver [response : struct mapped to type LoginResponse]
at CookComputing.XmlRpc.XmlRpcSerializer.ReportMissingMembers(Type valueType, Hashtable names, ParseStack parseStack)
at CookComputing.XmlRpc.XmlRpcSerializer.ParseStruct(XmlNode node, Type valueType, ParseStack parseStack, MappingAction mappingAction)
at CookComputing.XmlRpc.XmlRpcSerializer.ParseValue(XmlNode node, Type ValueType, ParseStack parseStack, MappingAction mappingAction, Type& ParsedType, Type& ParsedArrayType)
at CookComputing.XmlRpc.XmlRpcSerializer.ParseValue(XmlNode node, Type ValueType, ParseStack parseStack, MappingAction mappingAction)
at CookComputing.XmlRpc.XmlRpcSerializer.DeserializeResponse(TextReader trdr, Type returnType)
at CookComputing.XmlRpc.XmlRpcClientProtocol.ReadResponse(XmlRpcRequest req, WebResponse webResp, Stream respStm, Type returnType)
at CookComputing.XmlRpc.XmlRpcClientProtocol.Invoke(Object clientObj, String methodName, Object[] parameters)
at CookComputing.XmlRpc.XmlRpcClientProtocol.Invoke(String MethodName, Object[] Parameters)
at XmlRpcProxy1677f741-2cb9-462e-a585-95a3f1321775.Login(LoginParams )
at EF.ljArchive.Business.Sync.GoThread()

Win2000SP4, with no updates in sight on WindowsUpdate for .NET
(Reply) (Parent) (Thread)
From: gadsounet
2004-05-28 06:19 pm (UTC)
Sorry for the lag :|

It's a problem related to LJ Archive client, so you have to see it with fg. Sorry, I can't help... I don't even run XP and neither the .Net framework.

Thanks for testing!
(Reply) (Parent) (Thread)
[User Picture]From: alexmc
2004-05-06 02:38 am (UTC)
Well that sure is a coincidence. I was asking whether there was an LJ to PDF client just this last weekend.

(Reply) (Thread)
[User Picture]From: alexmc
2004-05-06 03:22 am (UTC)


If it is possible to get an XML extract of articles then has anyone tried something like LJ->XML->XSL:FO->PDF ?
That would be my method instead of LaTeX.

(Reply) (Thread)
From: gadsounet
2004-05-06 04:23 am (UTC)


First look for XSL:FO formaters on Google...
Then, figure out that there 2 free softwares (the other one are shipped at +$5000 (server version)).
Now, you are dealing with PassiveTex & FOP (from Apache group).
Look at some pages talking about them...

Now, consider my conclusions:
PassiveTeX: some problems with it... and it uses LaTeX...
FOP: Cool, the good way... but still not good enough...

Finally, fall back to LaTeX! :)

2 users pointed out XSL:FO earlier. That's right it would be the best solution. But up to now, I can't find a good XSL:FO formater which would allow me to do the same things as LaTeX (at the same price...).
If you find one, point it out, I would help to develop something using it.
(And yes, the option of paying $5000 has not be considered! :) )
(Reply) (Parent) (Thread)
[User Picture]From: alexmc
2004-05-06 06:25 am (UTC)


Aha, I follow your logic.

Personally I have some experience of Fop and no experience of LaTeX so my conclusion would have been diferent.

Goodluck and I hope I can help sometime.

(Reply) (Parent) (Thread)