?

Log in

No account? Create an account
RSS vs. Screenscraping - LiveJournal Client Discussions [entries|archive|friends|userinfo]
LiveJournal Client Discussions

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

RSS vs. Screenscraping [Dec. 3rd, 2002|07:46 am]
LiveJournal Client Discussions

lj_clients

[tcb]
This actually harkens back to gravitrue's post of Nov 13, but I felt that perhaps a new thread might be more productive.

Obviously the LJ implementation of RSS has a very critical missing component; comments.

I too would like to work on some sort of LJ->email gateway, and had hoped for something more elegant than what seems to be called screenscraping. Are there any LJ maintainers reading this forum? Do you have any insight into what the best way to handle this is? I'm sure this isn't a new problem..

If screenscraping is the only workable method, is there a preferred methodology that limits the hit on the servers as much as possible?
linkReply

Comments:
[User Picture]From: dottey
2002-12-03 01:57 am (UTC)
I don't understand. You want to grab comments from other blogs and put them into LJ? Or you want to grab LJ's comments and put them somewhere else?

Or is it unrelated to RSS, and you just want an email->LJ gateway (or LJ->email? which I don't understand) ?
(Reply) (Thread)
[User Picture]From: thelovebug
2002-12-03 02:13 am (UTC)
From what I can gather, I think Ted is referring to the Client/Server Protocol, rather than the RSS format.
(Reply) (Parent) (Thread)
[User Picture]From: tcb
2002-12-12 07:28 pm (UTC)
I should have checked back sooner..

What I want is an LJ->email gateway that allows for looking at a friends list, with comments.. read only. posting would be great, but that's a separate issue.
(Reply) (Parent) (Thread)
[User Picture]From: kimera
2002-12-03 05:11 am (UTC)
I think -- well, more like assume, which is dangerous, but I will go on -- he wants a way to update LJ via email. Blogger has this feature, though I wish you luck getting it to work. You setup an email address username.blogname@blogger.com and whatever you email to that gets updated.

Honestly, though, I'm not sure what that has to do with screen scraping or RSS (Blogger does use XML-RPC for it's clients), so I could seriously be off the mark. However, "LJ->email gateway" strikes me as wanting an LJ/email client.
(Reply) (Parent) (Thread)
[User Picture]From: benzado
2002-12-03 05:29 am (UTC)
The preferred method of screenscraping is to make a custom style designed for parsing, so you're not downloading a lot of HTML tags you don't need.

I really don't like the current situation, where the client developers are doing all sorts of weird tricks to get around limitations in the client-server protocol when it would make more sense just to update the server. The existing server developers are too busy with other things to take care of these requests.

I installed a copy of the LJ code on my machine, hoping to try to write a patch myself, but I haven't had time yet. Anyone who could spend the time to learn the server and write the patches would be my personal hero.
(Reply) (Thread)
From: evan
2002-12-03 07:09 am (UTC)
alanj (who does read this community) is the only person who really works on LJ full-time. So yeah, he's pretty busy. :)
(Reply) (Parent) (Thread)
[User Picture]From: benzado
2002-12-03 08:02 am (UTC)
I'm out of college and looking for work. If Brad wants to hire me, I'm game.
(Reply) (Parent) (Thread)
[User Picture]From: xb95
2002-12-03 08:13 am (UTC)
Get in line. ;) Heh. Seriously though, there are many people who would like to work for LiveJournal, including myself.
(Reply) (Parent) (Thread)
From: asciident
2002-12-03 09:52 am (UTC)
jproulx and sherm work full time for LJ, which obviously you know. ;P

Alan's just the person who's mainly looking at the server code.
(Reply) (Parent) (Thread)
[User Picture]From: xb95
2002-12-03 08:15 am (UTC)
RSS allows LiveJournal to get data, not give it. If you're wanting to download people's journals and the comments to go with them, you're talking about the client protocol. It doesn't exist, though you'd be welcome to code it up, and try to get it approved by bradfitz. I can assure you, it's very Fun and Exciting™.
(Reply) (Thread)
[User Picture]From: skywalker404
2002-12-12 10:43 pm (UTC)
I believe (if I understand the post correctly) I'm highly interested in building a client for what the poster is looking for. And since the protocol doesn't support retrieving entries from journals you can't log into (why?), it seems as though someone has to code it. I'd be interested in doing so, except I don't particularly know what language the LJ server system's in (BML?)...

Any information you, or others could provide would be definitely appreciated!
Thanks.
(Reply) (Parent) (Thread)
[User Picture]From: bradfitz
2002-12-15 03:50 pm (UTC)
since the protocol doesn't support retrieving entries from journals you can't log into (why?)

Because the first thing the protocol handler does is authentication. Keep in mind: the client/server protocol was originally designed for clients. If I needed it in my Win32 client, I added it. I never had a need for getting posts/comments of other journals.

except I don't particularly know what language the LJ server system's in

Perl.
(Reply) (Parent) (Thread)