[Rabbit-dev] bad url request error

[Romain Godefroy]

Hi,

Our rabbit proxy is used through sattelite links. With this kind of
network, the first TCP/IP packets are sometimes very small (96 bytes).
Then we got this exception:

30 janv. 2012 14:51:21 rabbit.filter.ProxyAuth getError
ATTENTION: Bad url: HTTP/1.1
java.net.MalformedURLException: no protocol: HTTP/1.1
at java.net.URL.<init>(URL.java:583)
at java.net.URL.<init>(URL.java:480)
at java.net.URL.<init>(URL.java:429)
at rabbit.filter.ProxyAuth.getError(ProxyAuth.java:96)
at rabbit.filter.ProxyAuth.doHttpInFiltering(ProxyAuth.java:53)
at
rabbit.proxy.HttpHeaderFilterer$InFilterer.filter(HttpHeaderFilterer.java:54)
at rabbit.proxy.HttpHeaderFilterer.filter(HttpHeaderFilterer.java:44)
at
rabbit.proxy.HttpHeaderFilterer.filterHttpIn(HttpHeaderFilterer.java:80)
at rabbit.proxy.Connection.filterAndHandleRequest(Connection.java:292)
at rabbit.proxy.Connection.access$400(Connection.java:47)
at rabbit.proxy.Connection$1.run(Connection.java:268)
at
org.khelekore.rnio.impl.StatisticsCollector.run(StatisticsCollector.java:37)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

I tried to add debug output in the HttpHeaderParser lineRead method.
When we got the error, the first line of the HTTP is troncated:

on= HTTP/1.1

instead of

GET
http://www.google.fr/m/search?q=test&pbx=1&aq=&oq=&aqi=&fkt=&fsdt=&cqt=&rst=&htf=&his=&maction=
HTTP/1.1

I check with a wireshark network capture and the request is complete.

I think the parser can't handle the first line because it is longer than
the first packet and the buffer read doesn't restart from begining?

Could you confirm our analyze? How should we fix that?
Thank you.

Romain Godefroy

[Romain Godefroy]

I suggest to add this code in HttpHeaderReader.java, after line 139:

// try to read some more
buffer.position (pos);
startParseAt = 0;

It seems to solve the problem.
Do you think it's a good patch?

Thank you

Post by Romain Godefroy
Hi,
Our rabbit proxy is used through sattelite links. With this kind of
network, the first TCP/IP packets are sometimes very small (96 bytes).
30 janv. 2012 14:51:21 rabbit.filter.ProxyAuth getError
ATTENTION: Bad url: HTTP/1.1
java.net.MalformedURLException: no protocol: HTTP/1.1
at java.net.URL.<init>(URL.java:583)
at java.net.URL.<init>(URL.java:480)
at java.net.URL.<init>(URL.java:429)
at rabbit.filter.ProxyAuth.getError(ProxyAuth.java:96)
at rabbit.filter.ProxyAuth.doHttpInFiltering(ProxyAuth.java:53)
at
rabbit.proxy.HttpHeaderFilterer$InFilterer.filter(HttpHeaderFilterer.java:54)
at rabbit.proxy.HttpHeaderFilterer.filter(HttpHeaderFilterer.java:44)
at
rabbit.proxy.HttpHeaderFilterer.filterHttpIn(HttpHeaderFilterer.java:80)
at
rabbit.proxy.Connection.filterAndHandleRequest(Connection.java:292)
at rabbit.proxy.Connection.access$400(Connection.java:47)
at rabbit.proxy.Connection$1.run(Connection.java:268)
at
org.khelekore.rnio.impl.StatisticsCollector.run(StatisticsCollector.java:37)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
I tried to add debug output in the HttpHeaderParser lineRead method.
on= HTTP/1.1
instead of
GET
http://www.google.fr/m/search?q=test&pbx=1&aq=&oq=&aqi=&fkt=&fsdt=&cqt=&rst=&htf=&his=&maction=
HTTP/1.1
I check with a wireshark network capture and the request is complete.
I think the parser can't handle the first line because it is longer
than the first packet and the buffer read doesn't restart from begining?
Could you confirm our analyze? How should we fix that?
Thank you.
Romain Godefroy

[Robert Olofsson]

On Mon, 30 Jan 2012 16:56:40 +0100

Post by Romain Godefroy
Our rabbit proxy is used through sattelite links.

Glad to hear you have found such a use for it.

Post by Romain Godefroy
With this kind of
network, the first TCP/IP packets are sometimes very small (96

Ok, that is interesting.

Post by Romain Godefroy
ATTENTION: Bad url: HTTP/1.1

Ok, I have seen one or two such issues, but not for some time.
What version are you using?

Can you try the current git version, the last changes
that could affect this are:

Date: Sun Jan 1 15:05:57 2012 +0100
Use the BufferHandler and CachingBufferHandler from rnio instead of
having our own copies.

Date: Sun Jan 1 16:20:33 2012 +0100
Try to make sure that growing buffers gets new content in the right
place.

Especially the second entry sounds like it could affect your
problem.

Can you do:
git clone http://www.khelekore.org/rabbit/rabbit.git
cd rabbit
make

If you do not have make you can build with ant instead.

Post by Romain Godefroy
I check with a wireshark network capture and the request is complete.

Ok, do you have that capture in a format that can be shared?
Be careful, it may contain sensitive information, but still
if I can see the traffic it might be easier to diagnose this.

Post by Romain Godefroy
I think the parser can't handle the first line because it is longer
than the first packet and the buffer read doesn't restart from
begining?

Could be, the code is supposed to try to read more in that case.

I do not think that your patch will work if you pipeline requests
though, so I do not think we can use it right away. Sadly the network
parsing is a bit complex to get right.

/robo

[Robert Olofsson]

Hi!

Thanks for the wire dump, that really helped me define some more tests
for the http header reader/parser. I managed to find and fix at least
one bug with partial headers, so hopefully everything is now working
well. Everything is in git for those that want to help test this.

/robo