may the force be with you

07 Mar 2006

The Java™ programming language has had an unprecedented run of success for vendors, customers, and the industry at large. But no programming language is a perfect fit for every job. This article launches a new series by Bruce Tate that looks at ways other languages solve major problems and what those solutions mean to Java developers. He first explores Active Record, the persistence engine behind Ruby on Rails. Active Record bucks many Java conventions, from the typical configuration mechanisms to fundamental architectural choices. The result is a framework that embraces radical compromises and fosters radical productivity.

2005 was in many ways a strange year for me. For the first time in nearly a decade, I began to do serious development in programming languages other than the Java language. While working with a startup, a business partner and I had some incredible success with a quick proof of concept. We were at a crossroads -- should we continue Java development or switch to something radical and new? We had many reasons to stay on the Java path:

• We'd need to learn a new language from scratch.
• The Java programming community was so strong that we'd have trouble getting our customer to accept a switch.
• We would not have the thousands of Java-centric open source projects to choose from.

About this series In the Crossing borders series, author Bruce Tate advances the notion that today's Java programmers are well served by learning other approaches and languages. The programming landscape has changed since Java technology was the obvious best choice for all development projects. Other frameworks are shaping the way Java frameworks are built, and the concepts you learn from other languages can inform your Java programming. The Python (or Ruby, or Smalltalk, or ... fill in the blank) code you write can change the way that you approach Java coding. This series introduces you to programming concepts and techniques that are radically different from, but also directly applicable to, Java development. In some cases, you'll need to integrate the technology to take advantage of it. In others, you'll be able to apply the concepts directly. The individual tool isn't as important as the idea that other languages and frameworks can influence developers, frameworks, and even fundamental approaches in the Java community.

But we didn't dismiss the idea of developing in another language and began to build our application in Ruby on Rails, a Web application framework built on the Ruby language. We were successful beyond our wildest imaginings. Since then, I've been able to split time between Java teaching and development (mostly with Hibernate and Spring), and Ruby teaching and development. I've come to believe that it's critical to learn other approaches and languages from time to time, for these reasons:

• Java is not the perfect language for every problem.
• You can take some new ideas back to your Java programming.
• Other frameworks are shaping the way that Java frameworks are built.

In this article, the first in a series that aims to demonstrate these ideas, you'll look at Active Record, the persistence architecture at the heart of Ruby on Rails. I'll also throw in a discussion of schema migrations.

Active Record: Radically different

When I took Rails out for a test drive, I freely admit my attitude was more than a little arrogant. I didn't think Active Record was up to the job, but I've since learned that it offers everything I need for some problems.

The best way to show you how Active Record handles persistence is to write some code. My examples use the MySQL database, but with minor changes, you can use the code with other databases as well.

Setting up Rails

By far, the easiest way to use Active Record is through Ruby on Rails. If you want to follow along in code, you need to install Ruby and Rails (see Resources).

Then, you can create a project. Go to the directory where you want Rails to create a new project, and type:

rails email_list

Rails creates your project, and you can use Active Record through some nifty Rails tools. The only step that remains is configuring your database. Create a database called people_development and edit the config/database.yml file to look like this (make sure you type in your username and password):

development:
adapter: mysql
database: email_list_development
host: localhost
username: root
password:

You needn't worry about the test and production environments in this article, so that's all of the configuration you need. You're ready to go.

Back to top

An example

In Hibernate, you'd usually begin development by working on your Java objects because Hibernate is a mapping framework. The object model becomes the center of your Hibernate universe. Active Record is a wrapping framework, so you start by creating a database table. The relational schema is the center of your Active Record universe. To create a database table, you can use a GUI or type this script:

CREATE TABLE people (
id int(11) NOT NULL auto_increment,
first_name varchar(255),
last_name varchar(255),
email varchar(255),
PRIMARY KEY (id)
);

Creating an Active Record class

Next, create a file called app/models/people.rb. Make it look like this:

class Person <>

This Ruby code creates a class called Person with a superclass called ActiveRecord::Base. (For now, assume Base is like a Java class and that ActiveRecord is like a Java package.) Surprisingly, that's all you need to do right now to get a good amount of capability.

Now you can manipulate Person from within the Active Record console. This console lets you use your database-backed objects from within the Ruby interpreter. Type:

ruby script/console.

Now, create a new person. Type these Ruby commands:

>> person = Person.new
>> person.first_name = "Bruce"
>> person.last_name = "Tate"
>> person.email = "bruce.tate@nospam.j2life.com"
>> person.save
>> person = Person.new
>> person.first_name = "Tom"
>> person.save

If you haven't worked with Active Record before, chances are you're seeing something new and interesting. This small example encapsulates two important features: convention over configuration and metaprogramming.

Convention over configuration saves you from tedious repetition by inferring configuration based on names you choose. You didn't need to configure any mapping because you built a database table that followed Rails naming conventions. Here are the main ones:

• Model class names such as EmailAccount are in CamelCase and are English singulars.
• Database table names such as email_accounts use underscores between words and are English plurals.
• Primary keys uniquely identify rows in relational databases. Active Record uses id for primary keys.
• Foreign keys join database tables. Active Record uses foreign keys such as person_id with an English singular and an _id suffix.

Convention over configuration gains you some additional speed if you follow Rails conventions, but you can override the conventions. For example, you could have a person that looks like this:

class Person <>

So convention over configuration doesn't unnecessarily restrict you, it just rewards you for consistent naming.

Metaprogramming is Active Record's other major contribution. Active Record makes heavy use of Ruby's reflection and metaprogramming capabilities. Metaprogramming is simply writing programs that write or change programs. In this case, the Base class adds attributes to your person class for every column in the database. You didn't need to write or generate any code, but you could use person.first_name, person.last_name, and person.email. You'll see more extensive metaprogramming as you read on.

Validating data

Active Record also includes some features that many Java frameworks don't have, such as model-based validation. Model-based validation lets you make sure the data within your database stays consistent. Change person.rb to look like this:

class Person <>

From the console, load person (because it's changed) and type these Ruby commands:

>> load 'app/models/person.rb'
>> person = Person.new
>> person.save

Ruby returns false. You can see the error message for any Ruby property:

>> puts person.errors[:email]
can't be blank

Back to top

Wrapping with relationships

So far, you've seen capabilities that you don't find in many Java frameworks, but you might not be convinced yet. After all, the hardest part of database application programming is often managing relationships. I'll show you how Active Record can help. Create another table called addresses:

CREATE TABLE addresses (
id int(11) NOT NULL auto_increment,
person_id int(11),
address varchar(255),
city varchar(255),
state varchar(255),
zip int(9),
PRIMARY KEY (id)
);

You've followed the Rails convention for your primary and foreign keys, so you'll get the configuration for free. Now change person.rb to support the address relationship:

class Person <>

And you should create app/models/address.rb:

class Address <>

I should clarify this syntax for readers new to Ruby. belongs_to :person is a method (not a method definition) that takes a symbol as a parameter. (Look at a symbol as an immutable string for now.) The belongs_to method is a metaprogramming method that adds an association, called person, to address. Take a look at how it works. If your console is running, exit it and restart it with ruby script/console. Next, enter the following commands:

>> person = Person.new
>> person.email = "bruce@tate.com"
>> address = Address.new
>> address.city = "Austin"
>> person.address = address
>> person.save
>> person2 = Person.find_by_email "bruce@tate.com"
>> person2.address.city
=> "Austin"

Before I talk about the relationship, look again at the findmethod, find_by_email. Active Record adds a custom finder for each of the attributes. (I've oversimplified things a bit, but this explanation works for now.)

Now, look at the last relationship. has_one :address adds an instance variable of type Address to person. The address is persisted as well; you can verify by entering Address.find_first in the console. So Active Record is actively managing the relationship.

You're not limited to simple one-to-one relationships, either. Change person.rb to look like this:

class Person <>

Make sure you pluralize address to addresses! Now, from the console, type these commands:

>> load 'app/models/person.rb'
>> person = Person.find_by_email "bruce@tate.com"
>> address = Address.new
>> address.city = "New Braunfels"
>> person.addresses <<>> person.save
>> Address.find_all.size
=> 2

The person.addresses <<> command adds the address to an array of addresses. Active Record added a second address to person, as expected. You verified that there was one more record in the database. So has_many works like has_one, but it adds an array of addresses to each Person. In fact, Active Record lets you have a number of different relationships, including these:

• belongs_to (many-to-one)
• has_one (one-to-one)
• has_many (one-to-many)
• has_and_belongs_to_many (many-to-many)
• inheritance
• acts_as_tree
• acts_as_list
• composition (mapping more than one class to a table)

From the very beginning, Active Record has helped to evolve my understanding of persistence. I learned that wrapping approaches aren't necessarily inferior; they're just different. I'd still lean on mapping frameworks for some problems, like crufty legacy schemas. But I think Active Record has quite a large niche and will improve as the mappings supported by Active Record improve.

I also learned that an effective persistence framework should take on the character of the language. Ruby is highly reflective and uses a form of reflection to query the definition of database system tables.

But, as you'll see now, my Rails persistence experience did not begin and end with Active Record.

Back to top

Evolving schema independently with migrations

As I learned Active Record, two problems plagued me. Active Record forced me to build create-table SQL scripts, tying me to an individual database implementation. Also, in development, I would often need to delete the database, which forced me to import all of my test data after each major change. I was dreading my first post-production push. Enter migrations, the Ruby on Rails solution for dealing with changes to a production database.

A migration example

With Rails migrations, I could create a migration for each major change to the database. For this article's application, you could have two migrations. To see the feature in action, create two files. First, create a file called db/migrate/001_initial_schema.rb and make it look like this:

class InitialSchema < limit =""> 255
    table.column "last_name", :string, :limit => 255
    table.column "email", :string, :limit => 255
  end
end
def self.down
  drop_table "people"
end
end

And in 002_add_addresses.rb, add this:

class AddAddresses < limit =""> 255
    table.column "city", :string, :limit => 255
    table.column "state", :string, :limit => 2
    table.column "zip", :string, :limit => 10
    table.column "person_id", :integer
  end
  Person.find_all.each do |person|
    person.address = Address.new
    person.address.address = "Not yet initialized"
    person.save
  end
end

def self.down
  drop_table "addresses"
end
end

You can migrate up by typing:

rake migrate

rake is like Java's ant. Rails has a target called migrate that runs migrations. This migration adds your entire schema. Go ahead and add a few people, but don't worry about the addresses yet.

Now suppose you've made a terrible mistake and need to migrate down to a previous version. Type:

rake migrate VERSION=1

That command took the first migration (denoted by the version number in the filename) and applied the AddAddresses.down method. Migrating up calls up methods on all necessary migrations, in numerical order. Migrating down calls the down methods in reverse order. If you look at your database, you'll see only the people table. The address has been removed. So migrate lets you move up or down based on your needs.

Migrations have another feature: dealing with data. You can migrate up again by typing:

rake migrate

This command runs AddAddresses.up and, in the process, initializes each Person object with an Active Record address. You can verify this behavior in the console. If you've added Person objects, you should also have Address objects. Open a new console and count the number of person and address database rows, like this:

>> Address.find_all.size
>> Person.find_all.size

So migrations deftly handle both schema and data. Now you can look at how these ideas translate to what's happening on the Java side.

Back to top

Java persistence

Java technology's persistence history is at once fascinating, tragic, and hopeful. Years of bad choices in the Java language's core persistence framework -- Enterprise JavaBeans (EJB) versions 1 and 2 -- led to years of struggling applications and disillusioned users. Hibernate and Java Data Objects (JDO), which both form the foundation of the new EJB persistence and a common persistence standard, led to a rise in object-relational mapping (ORM) within the Java community, and now the overall Java experience is a much more pleasant one.

Mapping

The Java community has had a seven-year love affair with ORM, a.k.a. mapping frameworks. Fundamentally, a mapping approach lets a user define Java and database objects independently and build maps between them, as in Figure 1:

Because the Java language is often an integration language first, mapping plays an important role for integrating crufty legacy systems, even those created long before any object-oriented languages existed. The Java community now embraces mapping to an incredible degree. Today, a typical Java programmer reaches for ORM to solve even basic problems. We like mapping because the Java mapping implementations often beat the Java implementations for alternative approaches: wrapping frameworks.

But you shouldn't disregard the power of wrapping frameworks. A wrapping framework places a thin wrapper around a database table, converting database rows to objects, as in Figure 2:

Your mapping layer carries a good deal of overhead if you don't need a map. And Java wrapping frameworks are seeing something of a resurgence. The Spring framework does JDBC wrapping and integrates features to do all kinds of enterprise integration. iBATIS wraps the result of a SQL statement instead of a table (see Resources). Both frameworks are brilliant pieces of work and underappreciated, in my opinion. But the typical Java mapping framework does things automatically that Java wrapping frameworks force you to do manually.

Java's needs

The Java platform already boasts state-of-the-art mapping frameworks, but I now believe that it needs a groundbreaking wrapping framework. Active Record relies on language capabilities to extend Rails classes on the fly. A Java framework could possibly simulate some of what Active Record offers, but creating something like Active Record would be challenging, possibly breaking three existing Java conventions:

• A persistence solution should work only on a Java POJO (plain old Java object). First and foremost, it would be difficult to create properties based on the contents of a database. A domain object might have a different API. Instead of calling person.get_name to set a property, you might use person.get(name) instead. At the cost of static type checking, you'd get a class built of metadata driven from a database.
  
  
• A persistence solution should express configuration in XML or annotations. Rails bucks this trend through forcing naming conventions with meaningful defaults, saving the user an incredible amount of repetition. The cost is not great because you can override defaults as needed with additional configuration code. Java frameworks could easily adopt the Rails convention-over-configuration paradigm.
  
  
• Schema migrations should be driven from the persistent domain model. Rails bucks this convention with migrations. The core benefit is the migration of both data and schema. Migrations also allow Rails to break the dependence on a relational database vendor. And the Rails strategy decouples the persistence strategy from the issue of schema migrations.

In each of these cases, Rails breaks long-standing conventions that Java framework designers have often held as sacred. Rails starts with a working schema and reflects on the schema to construct a model object. A Java wrapping framework might not take the same approach. Instead, to take advantage of Java's support for static typing (and the advantages of tools that recognize those types and provide features such as code completion), a Java framework would start with a working model and use Java's reflection and the excellent JDBC API to dynamically force that model out to the database.

RIFE with promise

One new Java framework that's moving toward sophistication as a wrapping framework is RIFE (and its subproject RIFE/Crud), created by Geert Bevin (see Resources). At the core, RIFE's persistence has three major layers, shown in Figure 3:

• A simple JDBC wrapper, which provides a callback-style implementation of JDBC through templates
• A set of database-independent SQL builders offering an object-oriented approach to query building
• A type-mapping layer that can convert most SQL types to most related Java types

These layers are used by simplified APIs, called query-managers, which provide intuitive access to persistence-related tasks such as save and update. One is specific to JDBC, and the other provides an analogous API that detects metadata related to RIFE's content-management framework.

RIFE/Crud sits on top of all of these frameworks, providing a very small layer that groups all of them together. RIFE/Crud uses constraints and bean properties to build an application's user interface, the site structure, the persistence logic, and the business logic automatically. RIFE/Crud relies heavily on RIFE's metadata capabilities to generate the interface and the relevant APIs, but it still functions with POJOs. RIFE/Crud is completely extensible through RIFE's clearly defined integration points in its API, templates, and component architecture.

The query-managers' API is strikingly simple. Here's an example of RIFE's persistence model in action. Suppose you have an Article class and you want to build a table structure around it and persist two articles to the database. You use this code to get a query manager, given a data source:

GenericQueryManager manager = GenericQueryManagerFactory.
  getInstance(datasource, Article.class);

Next, create the table structure in the database by installing the query manager:

manager.install();

Now you can use the query manager to access the database:

Article article1 = new Article("title");
Article article2 = new Article("othertitle");

manager.save(article1);
manager.save(article2);

List articleList = manager.restore(
manager.getRestoreQuery().where("title", "=", "othertitle"));

So, like Active Record, the RIFE framework uses convention over configuration. Article must support an ID property called id, or the user must specify the ID property using RIFE's API. And like Active Record, RIFE uses the capabilities of the native language. In this case, you still have a wrapping framework, but the model drives the schema instead of the other way around. And you still have a much simpler API than most object-relational mappers for many of the problems RIFE needs to solve. Better wrapping frameworks would serve Java well.

Back to top

Conclusion

Active Record is a persistence model written in a non-Java language that takes good advantage of that language's capabilities. If you've not seen it before, I hope this discussion has opened your eyes to what's possible in a wrapping framework. You also saw migrations. In theory, Java frameworks could adopt this concept. Though mapping frameworks have their place, I hope you'll be able to use this knowledge to look beyond the conventional mapping solutions in the Java language and catch the wrapping wave. Next time you'll take a look into continuation-based approaches to Web development.

Back to top

Resources

• Beyond Java (Bruce Tate, O'Reilly, 2005): The author's book about the Java code's rise and plateau and the technologies that could challenge the Java platform in some niches.
  
  
• "Rolling with Ruby on Rails" and Learn all about Ruby on Rails: Learn more about Ruby and Rails, including installation procedures.
  
  
• "Ruby on Rails and J2EE: Is there room for both?" (Aaron Rustad, developerWorks, July 2005): This article compares and contrasts some of the key architectural features of Rails and traditional J2EE frameworks.
  
  
• "Ruby off the Rails" (Andrew Glover, developerworks, December 2005): Andrew Glover digs beneath the hype for a look at what Java developers can do with Ruby, all by itself.
  
  
• Active Record: Active Record is the persistence framework for the Ruby on Rails framework.
  
  
• "Improve persistence with Apache iBATIS and Derby" (Daniel Wintschel, developerWorks, January 2006): Learn all about iBATIS, one of Java's best wrapping frameworks, in this three-part tutorial series.
  
  
• "The Spring series, Part 2: When Hibernate meets Spring" (Naveen Balani, developerWorks, August 2005): This article teaches the best combination for using Hibernate -- Spring with Hibernate.
  
  
• The Java technology zone: Hundreds of articles about every aspect of Java programming.

• The RIFE framework: RIFE uses some of the more radical techniques from non-Java languages.
  
  
• Ruby on Rails: Download the open source Ruby on Rails Web framework.
  
  
• Ruby: Get Ruby from the project Web site.
  
  

• developerWorks blogs: Get involved in the developerWorks community.
  
  

Back to top

About the author

		Bruce Tate is a father, mountain biker, and kayaker in Austin, Texas. He's the author of three best-selling Java books, including the Jolt winner Better, Faster, Lighter Java. He recently released Spring: A Developer's Notebook. He spent 13 years at IBM and is now the founder of the J2Life, LLC, consultancy, where he specializes in lightweight development strategies and architectures based on Java technology and Ruby.

# posted by dark master : 3/08/2006 09:30:00 AM