?

Log in

No account? Create an account
LJBook - LiveJournal Client Discussions [entries|archive|friends|userinfo]
LiveJournal Client Discussions

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

LJBook [Jan. 6th, 2004|06:53 pm]
LiveJournal Client Discussions
lj_clients
[gadsounet]
After playing some times with XML files because of a bad backup (thx ljpms :( ), I have made some proggies which create a PDF Book from your LJ's entries (and only them, no comments).

So if some people are interested:

LJ Book
(It is hosted on my DSL line down:512/up:128)

It should create a TEXT PDF Book from your entries in few minutes.
If it works fine, pictures will be included.

Thx to any tester to report any problems! :)
linkReply

Comments:
From: bobert225
2004-01-06 10:12 am (UTC)
I received a mass of segmentation fault errors, and then was redirected to an unviewable odf file due to those segfault errors.
(Reply) (Thread)
From: bobert225
2004-01-06 10:14 am (UTC)
err, pdf* file. I suck at typing today. >_
(Reply) (Parent) (Thread)
From: gadsounet
2004-01-06 10:26 am (UTC)
Segmentation fault ?! or LaTex Errors?!
Indeed I just got the log from pdfLaTeX which seems to have been in trouble with your LJ...
I will place a condition on my script which will enable me to store your XML files. Thus it will be easy for me to monitor what was going on.
So, if you are OK with this, come back to the page and relaunch the process. If not, thanks anyway! :)
(Reply) (Parent) (Thread)
From: bobert225
2004-01-06 10:44 am (UTC)
It was definitely a seg fault, but Moz Firebird wouldn't let me copy the error messages, or I'd have pasted them here. I'll relaunch the process now.
(Reply) (Parent) (Thread)
From: gadsounet
2004-01-06 11:52 am (UTC)
Thx! I found the bug... I have deleted your XML files from my computer.
(Reply) (Parent) (Thread)
[User Picture]From: tmtl
2004-01-06 10:44 am (UTC)
I tried logging in as the LJ test account (I don't know if I got the password correct...) and was presented with a blank PDF (There were headers and footers, and a couple of pages though) along with:

/home/zard/projects/ljml2tex/maketextbook.sh: line 17: 3124 Segmentation fault /home/zard/projects/ljml2tex/ljml2texo $i "/tmp/$1.xml"
/home/zard/projects/ljml2tex/maketextbook.sh: line 17: 3127 Segmentation fault /home/zard/projects/ljml2tex/ljml2texo $i "/tmp/$1.xml"
/home/zard/projects/ljml2tex/maketextbook.sh: line 17: 3140 Segmentation fault /home/zard/projects/ljml2tex/ljml2texo $i "/tmp/$1.xml"
/home/zard/projects/ljml2tex/maketextbook.sh: line 17: 3143 Segmentation fault /home/zard/projects/ljml2tex/ljml2texo $i "/tmp/$1.xml"


Which was printed right after </html> in the page source
(Reply) (Thread)
From: gadsounet
2004-01-06 10:46 am (UTC)
Thx! Same problem as for bober ... a buffer overflow :( Workign on it...
(Reply) (Parent) (Thread)
From: bobert225
2004-01-06 10:49 am (UTC)
I just relaunched the process. I received more errors, but the generated PDF had my entire journal properly displayed. The beginning looked empty, but when I scrolled down, I noticed that it had worked. One perfected, this is something you should propose that Brad includes in the site, so people don't worry about password security. It'll also be useful, because I know Support gets requests about how to download journal entries, and the answer's always been LJ's month-by-month method.
(Reply) (Parent) (Thread)
[User Picture]From: tmtl
2004-01-06 11:58 am (UTC)
Nice one

I now get the same iconv: illegal input sequence at position 33642 as ilmedic4life

This'll be really good once the bugs are ironed out

Would you ever consider releasing the source?
(Reply) (Parent) (Thread)
From: gadsounet
2004-01-06 12:05 pm (UTC)
iconv is used for the encoding from UTF-8 to LATIN-1
it's not a fatal error, and i will code my own converter, but it truncates the month processed...

The code will of course be released as GPL if it's useful, but it's a bit early...
(Reply) (Parent) (Thread)
From: crayolacrime
2004-01-06 11:08 am (UTC)
The PDF generated says 'Error' in a lovley font!

Awesome idea! I can't wait to see it working!
(Reply) (Thread)
From: gadsounet
2004-01-06 11:10 am (UTC)
This means the backend couldn't login... check login/pass.
(Reply) (Parent) (Thread)
[User Picture]From: adrieljohnson
2004-01-06 11:30 am (UTC)
While creating the book I got.

iconv: illegal input sequence at position 12884

Overall the book came out pretty good. I few things I did notice while reading it. No line preservations are taken into effect what so ever. And HTML does not show up. It looks really ugly. But this could be because of the iconv error. But other than the bugs the book turned out.
(Reply) (Thread)
[User Picture]From: codedragon
2004-01-06 11:47 am (UTC)
An excellent piece of work really. I'm very, very impressed.

Just a couple of points though: my LJBook appears to end in the middle of an entry. Also there seems to be limited support for line breaks (<br>) in entries, so that.

Lines
like

this

appear as:

Lines like this

Is this deliberate?
(Reply) (Thread)
From: gadsounet
2004-01-06 12:17 pm (UTC)
If it ends in the middle of an entry, it's certainly due to a iconv (encoding conversion) problem... don't know if XML files generated by LJ are purely UTF-8 or not ?!

and for the
problem, it's what I wanted (since I don't use HTML code in my entries). I have another piece of code which deals with HTML better (ie HTML Book on the site), but it doesn't look like TEXT Book... (it's no more a book for me indeed ;) ). It looks the same as if I generated a big HTML page with all your entries and you clicked on your browser Print Button. (such a way you could generate PDFs using Mozilla/Firebird ... )
(Reply) (Parent) (Thread)
From: gadsounet
2004-01-06 02:10 pm (UTC)
BR tags are no more dropped...
(Reply) (Parent) (Thread)
From: bobert225
2004-01-06 12:13 pm (UTC)
I'm just curious to what kind of interval you're using in querying the database through the client->server protocol. it seems like a very database intensive utility, and i'm all for things that don't put a huge load on LJ. :)
(Reply) (Thread)
From: gadsounet
2004-01-06 12:22 pm (UTC)
No interval... I just ask for each month of year 2003. I could place an interval, put the HTTP negociation is already one, isn't ?
I think, I do the same as the client "logjam" when it sync for the first time.
Perhaps it's too heavy ?! Nevertheless, it would be easy to restrict the access to 2 PDFs / year or month per user.
(Reply) (Parent) (Thread)
[User Picture]From: rog
2004-01-06 02:07 pm (UTC)
Cool, now I can produce my best-selling autobiography in just a couple of clicks! :)

Neat idea.
(Reply) (Thread)
[User Picture]From: lovewithnoface
2004-01-06 03:52 pm (UTC)
I don't know if it's worked yet, but I just need to say, LaTeX kicks serious butt!!!!

(Reply) (Thread)
[User Picture]From: lovewithnoface
2004-01-06 04:06 pm (UTC)
Got the following error:
/tmp/lovewithnoface_2003_08.xml:1002: error: Input is not proper UTF-8, indicate encoding ! os;t move my neck at all. So, instead of a lovely mixed set of soutenue and ch ^ /tmp/lovewithnoface_2003_08.xml:1002: error: Bytes: 0xEE 0x6E 0xE9 0x73 os;t move my neck at all. So, instead of a lovely mixed set of soutenue and ch ^ Document not parsed successfully. iconv: illegal input sequence at position 14796

Appears to be missing around a month of entries, but I can open the PDF file
(Reply) (Thread)
From: gadsounet
2004-01-07 05:38 am (UTC)
I would like to check out these XML files. I have modified my script in order to keep your backup. If you don't mind about it retry to generate a TEXT Book. Otherwise thx for testing! :)
(Reply) (Parent) (Thread)