S.E.A.N.I.C.U.S.

Monday, July 17, 2006

Ruby as Enterprise "Glue"

Any experienced Rubyist knows that the language can be used to create short, powerful shell scripts and integrate disparate systems, even if the community is just now realizing how powerful Ruby can be in the enterprise.

For example, today I was encountered with two problems. As a result of cleartext, unencrypted email addresses on the KCKCC website, many at the college are receiving lots of spam. We are in the process of obfuscating all of these addresses using a pretty standard Javascript function that composes the address from its parts. However, we have around 8000 items in our web directory structure, only some of which contain email addresses and only some of which contain cleartext ones (we've already had one pass at encrypting some). I thought immediately, "Ruby and Regular Expressions are the answer!" Around 30 lines later (including comments), I had a powerful Ruby script that could run from the shell and change all exposed addresses to their obfuscated counterparts.

However, several tests revealed that not all of the cases were covered. Luckily we have our webserver in a nightly-rsync backup configuration with another box, so I was able to test it on that box without breaking the website. As Eric and I tested the script and watched its output, we realized that even if the script works 90% of the time, there are occasionally going to be problems. We decided that it was long overdue to backup the website into a Subversion repository.

So I ran an svn import on the root web folder into a fresh repository. Little did I know that Apache/mod_svn has a 2GB limit on commits!
In the middle of adding files, it hung.

Ruby script to the rescue!

My first attempted solution was to add each top-level directory individually. Unfortunately, even some of those were over 2GB in size! So I modified the script to add and commit each file one-by-one, which quickly resulted in 900+ commits and it wasn't even finished. Then, I had an epiphany and realized I could keep track of how big each file was, and when I had accrued a large enough commit size, I could commit multiple files, thus saving in number of commits and transfer time (commit operations are atomic, thus expensive). I quickly reached the end of the day before I realized that I was committing the whole directory and not the accrued files, thus it had to recurse the whole directory structure to figure out what had been added. So I just left it to go, since it seemed to be doing fine, albeit having slow commits. Another caveat I found is that svn will not return an error code when something can't be added - only a warning.

Here's the code, roughly, for the final script (that I left running at the end of the day):
#!/usr/bin/env ruby
# usage: svnimport.rb [directory] [commitsize]
Dir.chdir(ARGV.shift)
@maxsize = ARGV.shift.to_i
@size = 0
Dir["./**/*"].sort.each do |file|
fork do
exec("svn add -N \"#{file}\"")
end
Process.wait
@size += File.size(file) if $? == 0
if @size >= @maxsize
fork do
exec("svn commit -m \"Initial commit\"")
end
@size = 0
Process.wait
end
end

Thursday, July 13, 2006

Potential DSLs

Sidebar: Yes I haven't posted here in a while -- I've been trying to get RadiantCMS to bend to my needs in terms of blogging, but that may be a ways off. We now return you to your regularly scheduled blog post...

One of the recent patterns with Rails has been to move most things into Ruby. Most recently, developers are encouraged (but not required) to move their database definitions into ActiveRecord::Migration format, both through a change in the default environment.rb file, and through automatic migration creation when generating a model. "Cool!" I said when I saw that.

In listening to this week's Ruby on Rails Podcast the issue of YAML came up, and it got me thinking. Yes, YAML is very simple and cleanly defines both the database configuration and fixtures, but couldn't the same thing be accomplished in Ruby? Jamis and others have encouraged the use of DSLs to simplify and clarify code. Since Django uses Python to describe its database configuration, why not use Ruby for Rails'?

I'm not sure whether I know how to accomplish it, but I'm going to try! Here's a sample of my proposed database configuration DSL:
common {
driver "mysql"
username "root"
password ""
host "example.com"
port "3306"
}

development {
<< "common"
database "app_development"
}

production {
<< "common"
database "app_production"
}

test {
<< "common"
database "app_test"
}
Since the specifying of common attributes and the "including" of them into other configurations seems popular, I decided to allow that with the "<<" operator, similarly to the way it's done in YAML.

The fixtures DSL would follow a pretty similar pattern, with the fixture name, followed with a block and corresponding attributes.

I'll probably post the code here when I finish prototyping it.

Your thoughts?

Thursday, June 22, 2006

New site is live

My new site, seancribbs.com is live! Today I got it hooked up with lighttpd so it runs MUCH faster. As soon as I get the weblogging functionality where I want it, I'll be moving my weblog over there. Until then, enjoy the rest of the site.

I heartily recommend Textdrive. The signup process was painless, the support ticket for a lighttpd port took less than 2 hours, and their servers are rock solid. Kudos to them!

Wednesday, June 21, 2006

Design Guidelines

Yesterday, Sierra (one of the WS interns) and I sat down in our usual way to work on a design mockup/comp for the website redesign. The way we decided to start this time was to look at a bunch college websites and critique them. We came up with a bunch of things that we liked and disliked about them and ultimately -- although it wasn't our intention -- some guidelines about what makes a good design in general. I'm sure this is documented more elegantly in many design books, but here's what we came up with. Keep in mind that none of the ideas are independent of one another, so I may repeat myself.

People-oriented


  • Information should be relevant to the user and organized and layed out in a person-friendly fashion. We may work with computers, but we don't work for them.
  • The design should have a personal touch, appealing to our sensibilities and evoking emotion. Many college websites had the "smiling faces" pictures which were effective.
  • Cater to your audience. The college sites we found most effective focused on who the user is, rather than how the colleges were organized.

Simplicity


  • Overall, the design should be clear and uncluttered.
  • Elements that have different meaning or function should be cleanly separated.
  • Less is more - say what you mean, briefly.

Focus


  • Our eyes are naturally drawn to the center. Put the most important information in the center, and use visual elements to draw our attention there.

  • Properly weight elements. Items that have more importance or hierarchical rank should appear larger or bolder or more saturated than ones with less significance. You'd be surprised how many times this principle is violated, even though it seems self-evident.

  • Avoid distractions. Don't put in a flashy, animated dancing carrot that draws the user's attention away from the primary content.

  • Make interface highlights (e.g. rollovers or page markers) subtle but meaningful.


Consistency


  • Make your primary and your secondary interface elements (e.g. front page and secondary pages) retain similarities so that the user doesn't have to learn a new interface every time.
  • Function should be correlated with appearance. If a button is in one interface, and has the same function in another interface, make it look the same.
  • Make your navigation consistent, for the love of Pete. This goes back to the first point.
  • I hate to borrow it too heavily, but use convention instead of configuration. In other words, give your user one way and one way only to achieve a certain result and make identical elements mean the same thing everywhere. The less they have to learn, the better.

Semantics


  • Make visual elements relevant to the context. If a page is about Faculty, have an image of an instructor in front of the classroom. If I'm on a page about International Student services, don't show me just any old picture of the campus.
  • Forge cognitive associations through consistency and iconography. Our brains largely discover meaning through association, so exploit it.
  • Don't be too wordy, use non-textual clues instead. It can be tiring to read a lot of text just to find out I'm on the Art department page. Show someone painting a canvas instead. This relates to simplicity and focus as well.


I never thought of myself as a designer, but I think I can tell a good design from a bad design. And now I have a rubrick to guide me in my own web designs!

A few personal notes

I decided to take the plunge and purchased some hosting from Textdrive this morning, and did a bunch of updates to my DynDNS account. Pretty soon, this blog will go away, in favor of my own site, the soon-to-be-registered seancribbs.com. My reasons for the change are these:
  1. I'm tired of Blogger's slow publishing time.

  2. I want more control over the layout, structure, and style of my blog.

  3. I want to be able to do web applications in a real environment, not just over my pitiful cable modem connection.

  4. Because I can.
The new site will have a template designed by me. Here's a preview. Yeah I know it's cheesy, get over it ;). Lately I have come to love the Franklin Gothic family of TrueType fonts, so if you have them, the site will be gorgeous. If not, you'll have to settle for Tahoma or Verdana. The site will also be running RadiantCMS, for which I intend to add a few modifications, including comments and integration with the Syntax library.

Tonight I'll be giving a presentation during the KCRUG Ruby on Rails workshop, specifically talking about AJAX and RJS with Rails. I'm in the midst of my preparations right now, so wish me luck!

KCKCC has made me an offer for at least a half-year, salaried and with benefits. I haven't decided whether I'll take it yet -- I may want to consult some people who have been in the workforce a bit longer. It's a decent offer, and while I don't want to stay there forever, it might be nice to use that time to improve my skills and get some more projects under my belt.