Author

Message

Rocket Man

Posts: 383

Posted on: 2012-11-13 16:26:48.000

It's not quite like winning an Olympic Gold Medal or anything, but it's a nice feeling.

I was having a very intermittent problem with long posts being truncated. Most posts are short enough that they didn't encounter this problem, but longer posts with code would sometimes just cut off randomly. The weird thing is that if I went back and refreshed the page they would go through. It wasn't predictable at all. Those kind of bugs are the most annoying!

Basically I had some old code that followed a similar technique that Dragonfly used:

(when (> (peek (device)) 0)
	(if (and (read (device) post-buffer $MAX_POST_LENGTH) post-buffer) ; grab all post data, put it in variable 'post-buffer'
		(parse-get-or-post post-buffer $POST)
	)
)

The idea is that you're reading from (device) into a variable called post-buffer. Hard to figure out what's going wrong in there!

Not sure how to fix this, I went and looked at the code in Artful Code's A Better newLISP Web Library: http://www.artfulcode.net/articles/a-better-newlisp-web-library/

The author had taken a slightly different approach, which I adapted into my framework:

(let ((buffer "") (post-buffer ""))
	(unless (zero? (peek (device)))
		(while (read (device) buffer $MAX_POST_LENGTH)
			(write post-buffer buffer))
		(parse-get-or-post post-buffer $POST)))

Basically, instead of just reading the (device) once, it CONTINUES to read the (device) and copies whatever it finds into a temporary variable "buffer" that gets appended to the variable "post-buffer" until there is nothing left to read.

It's nice when things get fixed like this. It makes me very happy.

Tiger

Posts: 9

Posted on: 2012-11-13 16:34:08.000

Yes, great feeling. I've noticed that "reading all at once" can fail silently; doing it in chunks with possibility of error recovery is more robust. Alternately, did you try a "catch" block, and figure out what the error was?

Also, I have the beginnings of a "mime" module; right now it parses MIME headers. It doesn't scoop in the data. Would you be interested in using it as a basis for parsing file uploads, etc? Although I think Lutz recently added file upload support to cgi.lsp So maybe I'm too late. But I do think MIME support is very useful on its own. As a serial protocol for transfering data with descriptive metadata it is very useful. It is so useful, I'm using my rudimentary mime module to control the freeswitch telephony server. Because freeswitch uses MIME for its command messages. Way more fun than XML.

Tiger

Posts: 9

Posted on: 2012-11-13 16:34:27.000

What I'm getting at here is, image uploads are broken. :)

Rocket Man

Posts: 383

Posted on: 2012-11-13 21:48:12.000

File uploads are next on my list. :) They work with Dragonfly, so they should work with Rockets... once I figure out how to parse binary data.

Tiger

Posts: 9

Posted on: 2012-11-13 22:38:34.000

Parse binary data? No, just do MIME, and read the binary data in, leaving it alone. And just keep searching for the boundary separator. Standard MIME.

I just noticed, the German online newspaper, der spiegel, has a forum feature for their blog posts; you read the blog article, and click to go to the forum. I wonder if they took the idea from you! I just noticed it tonight.
http://www.spiegel.de/international/

Rocket Man

Posts: 383

Posted on: 2012-11-14 15:45:29.000

Yeah, I don't actually need to parse the binary data, that was poor wording on my part.

Anyway I'm making my way through the MIME stuff and sort of figuring it out as I go, which is kind of fun. It's neat seeing the binary data for different types of files displayed as text... kind of like back in the days when I played with my very first HEX editor (ZAP, on CP/M, circa, oh, 1980?)

Rocket Man

Posts: 383

Posted on: 2012-11-16 17:02:31.000

Okay, it's pretty close to being done. I can upload files into a multi-part form and save them on the server. I just have to finish off the stuff where it grabs the filename, etc, and puts that into ($POST). We should have avatar uploads soon!

What I'd really like to do is also write a function (it will probably have to call some C functions or an external tool on the server) that can resize images, so when people upload GIANT PICTURES it sizes them properly. But that's something I can think about later. Other fish to fry!

Tiger

Posts: 9

Posted on: 2012-11-18 22:50:55.000

I'll be gone for a year. All the best, hope to have a beer with you then. Please submit a mime.lsp module to Lutz when you are able. Mime is so generally useful for data interchange! And since I won't be here to say this anymore, please please PLEASE spend 5-6 hours grokking Django. Rockets will be so much better for it. Just as newLISP is better because Lutz knew about modula and rexx.

Rocket Man

Posts: 383

Posted on: 2012-11-19 15:15:24.000

Best of luck with your trip, Tiger!