Log in

No account? Create an account
Problem with getfriends and a Ruby client - LiveJournal Client Discussions — LiveJournal [entries|archive|friends|userinfo]
LiveJournal Client Discussions

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Problem with getfriends and a Ruby client [Jun. 19th, 2002|11:06 am]
LiveJournal Client Discussions
[mood |puzzled]

I've been messing with Ruby a lot lately, and started working on a highly abstracted OO LJ library in it. However, I've hit a stumbling block with getting friends. The code is available for your perusal.

To use it, load up irb and run
load "lj.rb"
me = LJ::User::new("username", "password")

Right now that will output the HTTP response from the server along with any information relating to the actual function (for debugging purposes).

However, when I log in as myself, it throws an exception (one I made) and the data appears to be truncated.
irb(main):003:0> me.login
ProtocolError: ProtocolError

If I log in as test, it works (although test has no friends). It does work on an account with 13 friends, but not on mine with 83 or on another one with 161, so it seems to be related to the number of friends the user has.

I don't think there's a bug in my code, but I'd also be confused if it was a bug in Ruby's HTTP module. Unfortunately I can't (or don't know how to) get the HTTP headers Ruby is seeing. Any ideas? Has anyone else seen something like this?

Update: The location of truncation appears to be "random" - I just got
at the end instead. However, it always happens around that area (friend 20), which is a total of about 11 friends down my list (10-19,1,20-21). It happens in at the same place for the account with 161 friends, at friend 110 (100-109, 10, 110). So, whatever the problem is does appear to be tied to the data length somehow.

[User Picture]From: xb95
2002-06-19 10:01 am (UTC)
I don't know the language you're using, but I do have an educated guess as to what's happening based on looking at your code. It looks like you don't check for Content-Length, so you only get the first "write" and don't wait to see if there is more data. In HTTP, when there is a lot of data, things will fragment like:

Packet 1: HTTP header + some data
...your client gets this, sees that Length(some data) < Header.Content-Length, waits...
Packet 2: more data
...client sees Length(some data + more data) still < Header.Content-length, wait...
Packet 3: last data
...client notices Length(some data + more data + last data) == Header.Content-Length, returns data...

That's what it looks like is happening. You may want to check for the presence of Content-Length in the HTTP response header and then wait for that many bytes after the header before returning.

Good luck
(Reply) (Thread)
From: piman
2002-06-19 10:18 am (UTC)
I know about that header, but I'm using the language's standard library for HTTP requests, so in theory I shouldn't have to deal with that. If I do, I'd think there's a pretty big bug in Ruby's networking... If that turns out to be the case, I'll have to take it to ruby-talk.
(Reply) (Parent) (Thread)
From: piman
2002-06-19 10:27 am (UTC)
I changed the code in a way that should only be a semantic change (I changed it from code blocks to getting the return value), and now it works. This looks like it was a bug in Net::HTTP...
(Reply) (Parent) (Thread)
[User Picture]From: xb95
2002-06-19 10:54 am (UTC)
Yay for bugs. Report it to the Ruby folks and become rich and famous!

...or something. :)

Glad it's working though.
(Reply) (Parent) (Thread)
From: evan
2002-06-19 10:59 am (UTC)
Good job!

Here are some details:
The HTTP post method calls the read_body function, which is documented around line 282 in /usr/lib/ruby/1.6/net/http.rb:
: read_body( dest = '' )
    gets entity body and write it into DEST using "<<" method.
    If this method is called twice or more, nothing will be done
    and returns first DEST.
: read_body {|str| .... }
    gets entity body little by little and pass it to block.

Notice that it does different things for the block or for the string? Your block should work if you do

data = ""
lj.post("/interface/flat", data.to_cgi) do |str|
  data << str

and then pull every line below in one level of indentation.

This is actually sorta documented behavior:
The page's content is also passed to the << method of the dest parameter, or to the block if specified. This result is built network block by network block, not line by line.

So when you use the argument version of the post method, you are still getting the result block-by-block, but it's using the string's default << method so it appears it's getting the whole result.

(Reply) (Parent) (Thread)
From: evan
2002-06-19 11:00 am (UTC)
Erm, and clobbering your "data" variable was unintentional in my code sample. You should probably name it something different. :)
(Reply) (Parent) (Thread)
From: piman
2002-06-19 11:25 am (UTC)
I see. It makes sense, but it seems really un-Rubylike behavior.

Anyway, I guess the bug is in Ruby in a Nutshell for listing the two forms (blocks and return values) as equivalent.
(Reply) (Parent) (Thread)
From: evan
2002-06-19 11:41 am (UTC)
Well, they are equivalent, but it depends on how you interpret it: after the call to post is completed, both forms get all of the data.

It's just that within the block (and within the writing to the string in the other form) it happens in an iterative manner.
(Reply) (Parent) (Thread)
From: kap_x
2002-06-20 05:45 pm (UTC)

multiple clients

Does anybody know if/how I can run 2 or more Livejournal clients at one time? If so, please reply. That would be a huge help..thanks
(Reply) (Thread)
[User Picture]From: phil99
2002-06-22 03:38 pm (UTC)

Re: multiple clients

(Reply) (Parent) (Thread)