The newLISP on Rockets blog
Popular blog posts
Recent forum posts
Discussion Forum
Discussion forumCopying Apache log files to a database
Post #: 79
Post type: Blog post
Date: 2013-02-15 10:57:40.000
Author: Rocket Man
Apache log files typically just sit around filling up your hard drive space, a vast array of text too unwieldy to search through.
A useful thing to do is to parse these files and copy them to a database. I've written a script that will do this using Rockets. It's not a web page-- it's a script that you should run from the command line (use sudo newlisp log-to-database.lsp) It will create a new database from scratch called SERVER-LOGS.db that you can then query and search using SQLite.
The script can be found here: https://github.com/newlisponrockets/newLISP-on-Rockets/blob/master/log-to-database.lsp
Note: I may be updating this later, as the database files seems quite large (only slightly smaller than the logs themselves)
Views: 4513
How to implement New/Unread Posts in a forum
Post #: 78
Post type: Blog post
Date: 2013-01-16 14:14:11.000
Author: Rocket Man
This is a trickier issue than it appears at first.
The obvious solution is to just check the date of the last posted message and compare it with, say, the last time the user logged in. But in practice this doesn't work, because you want to view a post and have it removed from your "New" list right away, not at some later date. You actually have to bite the bullet and store a list of posts that you have read or unread. I chose "read" because by default users don't have anything in that list, which is easier to implement.
The list is stored as a string of post Id numbers, delimited by hyphens. When displaying the list of forum posts, it checks each one to see if the post Id is in your list of read posts, just by doing a (find) search on the string.
That part is easy. But what about when someone adds a new comment to a thread you've read? Ideally you want it to be bumped back to "Unread" status.
The solution I came up with is to get a list of every user's Read list whenever someone posts a comment. Then you loop through it and if you find the Post Id in there, you just snip it out using (replace).
Oh, I also added a "Mark All as Read" button, for people who just want all those pesky "New" icons to go away, RIGHT NOW. It gets a list of every post and adds them all to your Read list.
Is it as efficient as it could be? Will it bog down when you have a million users? I have no idea. I'll cross that bridge when I come to it. For now, it seems to work pretty well.
Views: 4730
View counts!
Post #: 77
Post type: Blog post
Date: 2013-01-15 17:19:09.000
Author: Rocket Man
I've added a small feature that shows the number of views each forum thread or blog post has gained. Each time you view a thread in the forum view or view a complete blog post by clicking on the link, the number is bumped.
Because this is a new feature, all the old views aren't counted. :( So they are starting at zero.
This did require a change to the database, as I didn't already have the column "PostViews" in the Posts table. This can present a problem when people are upgrading from an earlier version of Rockets that doesn't have that column. I've updated setup-rockets.lisp to create this field when making a new database, but people's existing databases won't have it. I'll probably have to add a "update database to latest version" script that will just add the missing columns for people with older versions. One nice thing is that the database doesn't need any values in there. It just needs to be altered to add this missing field.
Views: 4083
Happy New Year and a server upgrade!
Post #: 76
Post type: Blog post
Date: 2013-01-07 14:36:31.000
Author: Rocket Man
Welcome to 2013! It's fun living in the future.
I've upgraded the server from Ubuntu 10.04 to Ubuntu 12.04 LTS, so please let me know if there are any hiccups or strange behaviors on the site.
I'll be posting more soon!
Views: 9492
Towards the creation of newLISP on Rockets installation scripts...
Post #: 73
Post type: Blog post
Date: 2012-12-12 15:07:52.000
Author: Rocket Man
I'm now starting to use Rockets for projects at work, so I need a way to create the user database from scratch. Anyone wanting to start using the newLISP on Rockets Blog source code will need to do the same thing. So our desires align!
I've created a simple installation script called setup-rockets.lisp. It needs to be run from the command line:sudo newlisp setup-rockets.lisp
This will ask you a few questions (database, name, password, etc) and set up a new database for you with the right tables and create an admin user. Obviously you never want this to be run by someone else over the web, so make sure that it can't be viewed over your web server (either by permissions, or by using my Apache .htaccess directive that excludes all files ending in .lisp from being viewed (.lsp files are fine though!)
# Prevent framework source from being accessed
<Files ~ "\.lisp$">
Order allow,deny
Deny from all
</Files>
# Prevent database files from being accessed
<Files ~ "\.db$">
Order allow,deny
Deny from all
</Files>
Once the database is created (it will append the ".db" extension as well... you notice I've excluded those files so that you can't download the database from the web or via wget) you need to make sure it has the permissions to read and write by the owner:
chown www-data:www-data DATABASENAME.db
chmod 755 DATABASENAME.db
and you should be ready to go! Comments (20)
Views: 4886
An easy way to increase your post count...
Post #: 72
Post type: Blog post
Date: 2012-12-03 13:28:52.000
Author: Rocket Man
Is now to post more!
Now comments on blog posts, forum posts, or replies to forum posts will count towards your post count total. This feature wasn't implemented before, so I've gone ahead and retroactively counted everyone's posts (thanks to the COUNT(*) features of SQLite) to ensure not a single post of yours goes uncounted!
Post counts are a fun way to keep track of contributions to an online community. Implementing the +1 bump every time a user posts was quite easy:
; now update the user's postcount! postcount++!!
(set 'UserId Rockets:UserId)
(set 'UserPosts (++ Rockets:UserPosts))
(update-record "Users" UserId UserPosts)
I was pretty happy to be able to use the ++ function given that on many online forums, people post things like "Postcount++" when they post. Of course, this is LISP, so the ++ comes first.
You can view your post count by going to Edit Profile (click your name on the upper right to pull down the user menu) or simply by viewing a forum thread to see everyone's post counts! I didn't put post counts in the Blog view because blog posts are supposed to be, well, more about the post itself than about everyone increasing post counts. But if you view a blog thread in Forum view you will see that the appropriate counts have been updated.
At some point I will implement ranks based on postcount, which will also appear in the forum view and in your profile. So get out there and start posting! Comments (6)
Views: 4322
Calendars, date-pickers, and your birthday!
Post #: 69
Post type: Blog post
Date: 2012-11-27 15:14:43.000
Author: Rocket Man
Happy Birthday! Or is it?
It's handy in web applications to have a way to select dates in a rigid format, and to have a little calendar to make it easier for users to select dates. I've incorporated a plugin for Bootstrap that does just that.
Inside a form, just use the following code:(form-datepicker "Enter your birth date" "date" show-birthdate "dp1")
This will open up a little form box where you can type in a date, or if you click on the date box a calendar pops up.
To see it in action, go to your User Profile page (click on your name on the upper right) and you can set your birthdate there! At some point I'll add a little feature that will send you a Happy Birthday email, just for fun.
When I was implementing this I discovered that SQLite wants dates in a YYYY-MM-DD HH:MM:SS.000 format and this calendar wants it in MM-DD-YYYY format. It's quite easy to convert between the two, but I might add some date functions to Rockets to make this easier. Of course everyone has their own preferred date format, which complicates things still further...
Views: 4009
Avatar uploads are working!
Post #: 68
Post type: Blog post
Date: 2012-11-19 20:52:42.000
Author: Rocket Man
After a fair bit of fiddling, I've gotten avatar uploads working on the newLISP on Rockets blog. Now you can upload your own image.
The code to display a file upload form looks like this:
(displayln "<p>Upload new avatar (all avatars scaled to 64x64 pixels): <form name='FileUpload' action='rockets-avatarupload.lsp' method='POST' enctype='multipart/form-data'><input type='file' id='uploadName' name='uploaded_data' onChange='this.form.textname.value = this.value'><input type='hidden' name='textname'><input type='submit' value='Upload' name='submit'></form>")
The form variables are put into the ($POST) tree, along with a variable called "filename" that holds the file name, and "binary-data" that holds the file itself.
To retrieve these from ($POST), just use the following code:
(set 'file-name ($POST "filename"))
(set 'file-binary ($POST "binary-data"))
Then to save the file , I use the following command:
(write-file (string "images/avatars/" file-name) file-binary)One bit of warning: you will need to set the ownership permissions on the directory where you are saving these files to something that the Apache user has write access to. In Ubuntu this was something like:
sudo chown www-data:www-data images/avatarsAnd that's it! The avatar images are not scaled when they are uploaded, but they are displayed with fixed dimensions of 64x64 pixels. This means if you upload a larger image it will look tiny. In the future I will add a module that automatically resizes files to a dimension of your choice, but for now, try to keep avatars to no larger than 100x100. Comments (2)
Views: 4251
The feeling of triumph when fixing a stubborn bug...
Post #: 67
Post type: Blog post
Date: 2012-11-13 16:26:48.000
Author: Rocket Man
It's not quite like winning an Olympic Gold Medal or anything, but it's a nice feeling.
I was having a very intermittent problem with long posts being truncated. Most posts are short enough that they didn't encounter this problem, but longer posts with code would sometimes just cut off randomly. The weird thing is that if I went back and refreshed the page they would go through. It wasn't predictable at all. Those kind of bugs are the most annoying!
Basically I had some old code that followed a similar technique that Dragonfly used:
(when (> (peek (device)) 0)
(if (and (read (device) post-buffer $MAX_POST_LENGTH) post-buffer) ; grab all post data, put it in variable 'post-buffer'
(parse-get-or-post post-buffer $POST)
)
)
The idea is that you're reading from (device) into a variable called post-buffer. Hard to figure out what's going wrong in there!
Not sure how to fix this, I went and looked at the code in Artful Code's A Better newLISP Web Library: http://www.artfulcode.net/articles/a-better-newlisp-web-library/
The author had taken a slightly different approach, which I adapted into my framework:
(let ((buffer "") (post-buffer ""))
(unless (zero? (peek (device)))
(while (read (device) buffer $MAX_POST_LENGTH)
(write post-buffer buffer))
(parse-get-or-post post-buffer $POST)))
Basically, instead of just reading the (device) once, it CONTINUES to read the (device) and copies whatever it finds into a temporary variable "buffer" that gets appended to the variable "post-buffer" until there is nothing left to read.
It's nice when things get fixed like this. It makes me very happy. Comments (8)
Views: 4982
MVC code patterns, templates, and complexity
Post #: 65
Post type: Blog post
Date: 2012-11-08 15:15:49.000
Author: Rocket Man
Today I implemented an entirely different way to view posts and comments, the forum view. When you view a blog post, the same post also exists in the forums. In fact, you can click on the "View post in forums" button to toggle the views.
A lot of people would tell you to write this kind of thing using the MVC (model-view-controller) model, or at least use a framework that has a templating system so that you can view the same data differently.
This is a completely valid perspective and I'm not going to argue otherwise. MVC is a well-established design methodology and it works pretty well for a lot of different applications. Templates seem like they could be really useful.
I didn't implement the forum view this way, however. I just had a toggle variable called (forum-view-post) and checked to see if it was true. If it was, then I executed a different code block. The real danger here is code duplication, which is something we always want to avoid.
The thing is, implementing this very different view took only ten lines of code.
(displayln "<h3>" (list-post-data 3) "</h3>")
(set 'header-list '("Author" "Message"))
(set 'post-data (list (string "<img src='images/avatars/" (author-avatar (list-post-data 1)) "'><br>" (author-name (list-post-data 1))) (format-for-web (list-post-data 4))))
(set 'PostId (int (list-post-data 0)))
(set 'post-data (list post-data)) ; okay these two lines of code are duplicated... I can live with it for now
(set 'post-comments (get-record "Comments" PostId))
(if post-comments (begin
(dolist (p post-comments)
(push (list (string "<img src='images/avatars/" (author-avatar (p 2)) "'><br>"(author-name (p 2))) (format-for-web (p 5))) post-data -1)) ; add each comment to the thread
))
(display-table header-list post-data "striped")
Okay, so two lines of code are duplicated in the alternate (blog) view, which I had already written. If it was much more than this, maybe I'd write a function and move it outside the block so that I didn't have duplicate code. Obviously I don't want to have to change things multiple times every time I change the view. But these two lines only extract the comment data from the database, so I wouldn't be changing these anyway.
Now, let's say I wrote this using a template system, or with MVC, or both. It would be a lot more than ten lines of code. I would have to learn another language (the templating language) and I would have to write a bunch of classes and methods and make sure they all work properly together, maybe writing a bunch of unit tests to ensure I don't break anything if I change the contracts between the model, view, and controller. More code equals more opportunities for bugs. In fact, the number of lines of code is the only aspect of development that has a strong correlation with the number of bugs. Everything else is too hard to measure. Fewer lines of code also take less time to write, in general. So all else being equal, wouldn't you want to have fewer lines of code to do the same thing?
As far as templates go, they make a lot of sense in a big company where there is a whole department doing user interface design and another department doing coding. The designers can learn the templating language and HTML, and not have to learn anything else. For my personal development, I am both the interface designer and the developer.
The newLISP code is my templating language. I don't want to have to learn another language if I can just keep everything together. Overall, the arguments for using MVC and templating work better for larger projects with lots of people. newLISP on Rockets is optimized for very small development teams, including projects where the team is only one person. In these cases, simplicity almost always wins out over complexity. Comments (9)
Views: 4670