images images images

Last October I began working on a site that crawls a bunch of different job boards looking for remote job posts. I wanted to see if I could go from literally nothing to a fully-functional, useful site in 30 days on my own free time.

I decided on this particular project for an odd reason: lots of other people were building nearly identical sites. You might think that’d be discouraging, but I decided that proved it’d be pretty trivial to do, and that there are definitely people out there who would find it useful. I have no delusions about this site making me rich, so it’s no big deal if there are tons of competitors out there.

In the past I’ve used project sites like this to learn new technology. I hemmed and hawed for a long time about whether I wanted to use this site as a change to learn Elixir+Phoenix and/or ReactJS. In the end I decided that the challenge was to build something in 30 days, and my track record with previous “learning” projects were that, while I’d learn something, they’d almost never get finished. The extra effort of teaching myself something new made it harder to finish. The goal for this project was explicitly not to learn something new in terms of technology, it was to see if I really could see a project through from start to finish given the fewest impedements. It turns out I can!

RemotelyAwesomeJobs is thus unsurprisingly built on Rails 4.2 with Postgres and Sidekiq in support. There’s minimal Javascript (I decided that might actually be a virtue in this case – I want something that works on any device with little fuss). I did decide to host it myself on DigitalOcean instead of going the stupid-easy Heroku route, mostly because it’s a lot cheaper to do so once you want more than one server.

Anyhow, check it out if you’re looking for a remote job. I’m “dog- fooding” it myself right now, so it should be pretty stable and usable at this point. ;)

Link: https://www.remotelyawesomejobs.com

(This post originally appeared on Burnside Digital’s blog on July 12, 2013. Alas, Burnside Digital’s blog is no more, so I’ve reposted it here.)

We recently had a project for a client where we wanted to eagerly load a model’s associations, but only under certain conditions – the associated models were rendered from a page fragment that was cached. It turns out this is pretty easy to do.

The project that we needed this solution for has a basic CMS. The schema for the CMS looks something like this:

In other words, our queries were fraught with opportunities for gobs and gobs of dreaded “N+1 queries.” To avoid the n+1 queries, we could simply eagerly load by invoking include:

1
2
3
4
5
6
7
8
9
class HomeController < ApplicationController
  def index
    @homepage = HomePage.find(params[:id], :include => 
      {:page_section =>
      {:page_section_schedule =>
      {:page_fragment =>
      {:carousel_slide => :image}}}})
  end
end

However, there was caching to consider – this data rarely changes (at most, daily), so it’s also a great candidate for fragment caching. With a fragment cache in place, the page looks a bit like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
<html>
 <head>
  <title>Home Page</title>
 </head>
 <body>
  <h1>Home Page</h1>
  <div>
    <%- cache( "homepage-fragment-#{@homepage.cache_key}" ) do %>
      <%= render "content", :locals => { :homepage => @homepage } %>
    <%- end %>
  </div>
 </body>
</html>

With fragment caching, all that laborious eager loading isn’t necessary – the associations aren’t needed because the data is already cached in a fragment.

To get around this, we simply perform the eager loading only when we need it, in the cached fragment:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<html>
 <head>
  <title>Home Page</title>
 </head>
 <body>
  <h1>Home Page</h1>
  <div>
    <%- cache( "homepage-fragment-#{@homepage.updated_at}" ) do %>
      @homepage.do_eager_loading
      <%= render "content", :locals => { :homepage => @homepage } %>
    <%- end %>
  </div>
 </body>
</html>

The do_eager_loading method uses Rails’ internal eager loading methods to get the work done. It looks like this:

1
2
3
4
5
6
7
8
9
10
11
class HomePage < ActiveRecord::Base
  has_many :page_section_schedules

  def do_eager_loading
    ActiveRecord::Associations::Preloader.new([self], 
      {:page_sections => 
      {:page_section_schedules => 
      {:page_fragment => 
      {:carousel_slides => :image} }}}).run
  end
end

And voila – if the content is cached, no eager loading takes place. If the cache isn’t populated or is invalid, we eager load.

(This post originally appeared on Burnside Digital’s blog on July 12, 2013. Alas, Burnside Digital’s blog is no more, so I’ve reposted it here.)

Recently, a colleague who is a front-end developer asked me what the qualities were for a good REST API. He had been experimenting with AngularJS and wanted to get a rough idea of whether or not he could expect the resource objects returned by Angular’s $resource factory to work nicely with them.

Strangely, this was a question hard to pin down an answer for. I’ve worked with RESTful APIs for years and had a good idea what made a good REST API (“I know one when I see it”), but never really given much thought to it. Roy Fielding first defined REST in his doctoral dissertation back in 2000, but that tome was a lot thicker than the quick bullet-points my colleague was looking for.

RESTfulness is something that happens in degrees. I think I might classify it thusly:

Not RESTful, even though the API creators might claim that it’s RESTful These alone don’t make an API RESTful:

  • Uses HTTP or HTTPS
  • Returns XML and/or JSON

Occasionally you’ll run into some misguided folks who think that the above two points are all it takes to slap the REST seal-of-approval on their APIs. Ironically, REST pedants will tell you that specifying the protocol or format not only is inadequate qualification for RESTfulness, but it actually runs afoul of the spirit of REST.

Warning signs that a self-proclaimed REST interface that isn’t:

Mostly RESTful

A mostly (and I’d claim adequately) RESTful interface has these qualities:

  • Uses URLs to refer to resources. These resources are the “nouns” in the system. Examples of a resource might be an account, a transaction, a user, an appointment, etc.
  • Uses the four best-known, well-recognized “HTTP Verbs”, and defines them semantically to mean the following “CRUD” operations:
  • GET – read a resource or list of resources
  • POST – create a new resource
  • PUT – modify an existing resource
  • DELETE – delete a resource
  • You might wonder how you actually do something like, e.g., a payment, in a system designed so noun-centrically like this. It seems actions/verbs have no place in it. However, even in these cases you are creating a transaction. So you’d POST/create a payment resource. Everything is expressed in terms of resource lifecycle events.
  • Uses semantically appropriate error codes. E.g., 404 means the resource isn’t found. A 401 means you aren’t authorized to view it. A 422 means something about the data or request was malformed.

Hardcore RESTful / Hypermedia

The highest degree of RESTful APIs I’ll refer to as “Hardcore RESTful.” They have the following qualities:

  • Doesn’t necessarily define a protocol/transport (e.g., HTTP). As I said above, pedantic REST nerds will tell you that restricting REST to a single protocol violates the spirit of REST. The reality is that REST almost always implies HTTP.
  • Doesn’t define something like a version number in the URL – a version isn’t semantically a resource or a part of the resource. Instead API versioning can be done using something like HTTP header values (e.g., Accept:), or some other side-band portion of the protocol.
  • Similarly, it shouldn’t define a desired data format in the URL (e.g., http://example.com/foo.json), and should instead do that via some side-band part of the protocol like HTTP headers (e.g., content Accept:). You can embed versioning and desired format in a MIME type that is passed as an Accept header like this: application/vnd.mycompany.myapp-v1+json
  • To take a RESTful interface up to the next level, you can create an API that builds upon REST and does not return hierarchical data. Instead it returns URLs to subsets of data. This style of API is usually referred to as a “Hypermedia API” for its reliance on hyperlinks. E.g.,

BAD

1
2
3
4
5
6
7
8
9
10
<account>
  <name>Checking</name>
  <balance>100.00</balance>
  <user>
    <name>Elvis Presley</name>
    <address>Graceland</address>
    <city>Memphis</city>
    <state>TN</state>
  </user>
</account>

GOOD

1
2
3
4
5
<account>
  <name>Checking</name>
  <balance>100.00</balance>
  <user href="http://example.com/user/elvis-presley"/>
</account>

The argument for this style of design are that your inner URLs can change at a later date, and not break the entire API. It also enables “discovery” of the interface by allowing a client to start at a root node of the API, following links much like a search-engine spider or bot would. This also engenders systems to evolve their APIs independently -theoretically, a client with a generic ability to interpret hypermedia shouldn’t get tripped up by a less rigid definition of the API. Proponents of Hypermedia APIs often speak of a concept called “Hypermedia as the Engine of Application State“, or HATEOAS for short.

Arguments against this style is that, in practice, this manner of API discoverability is of limited usefulness and actually incurs an overhead by requiring more API calls than would be otherwise needed.

In Summary

What I told my colleague was this: most REST APIs out there meet the “mostly RESTful” level of compliance. Given that, most REST client abstractions will probably work OK if the API is at least at that degree of RESTfulness (with the caveat that I didn’t wade very deeply into Angular’s resource library!). If you’re creating a brand new REST API, it’s good to aim a little higher than that – you’re clients will appreciate it.

If you want to read more about REST and Hypermedia APIs, Steve Klabnik has written an excellent series of blog posts
which expand on these ideas a whole lot more.

For a recent work project at Openbay, we decided to use Resque for our delayed job processing (might move to Sidekiq eventually, but it’s not a priority right now). That meant we had a Redis server already up and running in our environment. When it came time to implement partial caching, we knew that memcache would probably have the best performance, but we already had Redis kicking around anyway, so we figured we’d try using Redis for it instead.

What we forgot to do was to namespace our partial cache. This meant that all of our partials were stored in the same root namespace with everything else, including our delayed jobs.

We eventually noticed that queued jobs were never getting kicked off. While diagnosing the problem, I decided I’d try clearing the cache (for unrelated reasons). Imagine my horror:

1
2
3
4
5
6
7
irb(main):005:0> Resque.delayed_queue_peek(0,1000)
=> [1393632000, 1393635600, 1393639200, 1393696996, 1393696998, 1393697211, 1393697222, 1393699884, 1393718400, 1393783396, 1393783398, 1393783611, 1393783622, 1393786284, 1393869796, 1393869798, 1393870011, 1393870022, 1393956196, 1393956198, 1393956411, 1393956422, 1393959084, 1394042596, 1394042598, 1394042811, 1394042822, 1394045484, 1394128996, 1394128998, 1394129211, 1394129222, 1394131884, 1394330596, 1394330598, 1394330811, 1394330822, 1394333484, 1394733796, 1394733798, 1394734011, 1394734023, 1394736684, 1395050596, 1395050598, 1395050811, 1395050823, 1395053484]
irb(main):006:0> Rails.cache.clear
=> "OK"
irb(main):007:0> Resque.delayed_queue_peek(0,1000)
=> []
irb(main):008:0>

ZOMG. Rails.cache.clear was wiping out all of our delayed jobs!

The fix was simple enough – namespace your cache entries in your environment.rb file:

1
  config.cache_store = :redis_store, ENV['REDIS_URL'], { :namespace => 'production-cache' }

Don’t forget to namespace! After that, we were all better:

1
2
3
4
5
=> [1393617439, 1393632000, 1393635600, 1393703632, 1393718400, 1393790032, 1393962832, 1394049232, 1394135632, 1394337232, 1394740432, 1395057232]
irb(main):006:0> Rails.cache.clear
=> 20
irb(main):007:0> Resque.delayed_queue_peek(0,1000)
=> [1393617439, 1393632000, 1393635600, 1393703632, 1393718400, 1393790032, 1393962832, 1394049232, 1394135632, 1394337232, 1394740432, 1395057232]

I spent the better part of last night trying to get Amazon SES working from my Mac. After reading the README, installing all the dependencies, and doing everything correctly, I kept getting the following error:

Can't locate object method "ssl_opts" via package "LWP::UserAgent" at SES.pm line 249.

The only Google hits were for people trying to set up SES with Ubuntu or Debian and weren’t much help. It finally dawned on me what was going on this morning… when I did a “which perl”, I noticed that it was running from my MacPorts directory:

~/Downloads/amazon-ses> which perl
/opt/local/bin/perl

However, when I looked at the code in the script, the “she-bang” at the top of the script was pointing to the perl in /usr/bin, which was installed with the operating system:

#!/usr/bin/perl -w

So all of my dependency installations were ending up in my MacPorts version of Perl, but the script was executing using the OSX version of Perl, and nothing worked. Changing the first line to

#!/opt/local/bin/perl -w

fixed the problem for me. It probably would make even more sense for me to make /usr/bin/perl a symbolic link to my MacPorts installation. I hope this can help someone else out there suffering through the same problem!

A little over a year ago, I inherited someone else’s JRuby on Rails project. My initial role on the team was the ‘token Java guy,’ and knew almost zero about Ruby. I’ve since grown to love JRuby, but that’s a story for another time.

Over the course of the project, I’ve upgraded my JRuby version, upgraded gems, upgraded Java, and who knows what else. One day, I went to do a migration, and they just plain didn’t work anymore. I discovered that most migration-related stuff didn’t work, including tasks like db:test:load. And by “didn’t work anymore,” I mean this:

Meep:~/_work/sonymusic/src/checkout> rake db:test:load
(in /Users/mdesjardins/_work/sonymusic/src/checkout)
Invalid access of stack red zone 0x100401b20 rip=0x10a6ceb6e
Bus error

My initial reaction to this is what I expect most people’s reaction would be: “what in holy hell is this?” It’s basically a blown stack – the stack has “edges” called red zones to which you should never write, because it means you’re getting too close to going off the end, and the OS is trying to prevent you from being stupid.

Google was little help – most people speculated that similar errors were problems w/ OSX’s JVM implementation.

So I flailed for a while. I tried backing up to version 1.5 of Apple’s JVM (which itself is nontrivial, and was made more complicated by the fact that json-jruby requires Java 6, but I digress). When that failed, I tried rolling back random gems and versions of JRuby. I tried enabling core files and reading stacks in gdb, but that ended up being useless. I tried increasing the stack to a whopping 2 Gig (normal is 2M) by passing -J-Xss2048M, but it still crashed.

When I passed -J-d32 on JRuby’s command line, I got a nice “Stack Limit Exceeded” error instead of the nasty crash – so that was a start. Everything pointed to an infinite recursion problem, but where?

I was getting desperate. As it turns out, before I became a retread Ruby programmer, I worked in Java enterprise stuff. This is the kind of stuff you do at banks and insurance companies, in hellish beige cubicles, where you hang posters with diagrams of middleware and service busses and ORMs.

In beige cubicle hell, I learned about a few tools to monitor active JVMs (this comes in handy when you’re monitoring god-awful beasts like “enterprise application servers.”) I figured it was worth a shot to try out jstack. jstack comes with the JVM. It shows you a stack dump of all the threads running in a JVM. If you’re using it on a local JVM, all you need to do is pass it the PID of the process you’d like to observe w/ the -l (for local?) command line switch.

The trick was going to be catching it just before it blew up. After a few tries, I caught this:

"main" prio=5 tid=102801000 nid=0&#215;100601000 runnable [10047d000]   java.lang.Thread.State: RUNNABLE 
at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:123) 
at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:188) - locked <7fd39d7c8> (a com.mysql.jdbc.util.ReadAheadInputStream) 
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2329) 
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2774)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2763)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3299)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1837)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1961)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2537) - locked <7fd3992c0> (a java.lang.Object)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2466)
at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1383) - locked <7fd3992c0> (a java.lang.Object)
at com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:4815)
at com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50)
at com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4793)
at arjdbc.jdbc.RubyJdbcConnection$16.call(RubyJdbcConnection.java:1015)
at arjdbc.jdbc.RubyJdbcConnection.withConnectionAndRetry(RubyJdbcConnection.java:1191)
at arjdbc.jdbc.RubyJdbcConnection.tables(RubyJdbcConnection.java:576)
at arjdbc.jdbc.RubyJdbcConnection.tables(RubyJdbcConnection.java:552)
at arjdbc.jdbc.RubyJdbcConnection$i$tables.call(RubyJdbcConnection$i$tables.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:103)
at rubyjit.tables_AA8BB5A33C226C4FACC724EF93DE8D1DF04792EB.__file__(/Users/mdesjardins/.rvm/gems/jruby-1.6.0@rails238/gems/activerecord-jdbc-adapter-1.1.1/lib/arjdbc/jdbc/adapter.rb:234)
at rubyjit.tables_AA8BB5A33C226C4FACC724EF93DE8D1DF04792EB.__file__(/Users/mdesjardins/.rvm/gems/jruby-1.6.0@rails238/gems/activerecord-jdbc-adapter-1.1.1/lib/arjdbc/jdbc/adapter.rb)
at org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:37)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:127)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:103)
at rubyjit.auto_create_table_E1FB7FE0CE1D294ABA67860B81492049B1D6AE7F.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb:70)
at rubyjit.auto_create_table_E1FB7FE0CE1D294ABA67860B81492049B1D6AE7F.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:87)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:78)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)
at rubyjit.method_missing_with_auto_migration_CA4124448766A2D0B44A8F3F5A4A9EC9908F2D02.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb:55)
at rubyjit.method_missing_with_auto_migration_CA4124448766A2D0B44A8F3F5A4A9EC9908F2D02.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:87)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:148)
at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:101)
at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:101)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:78)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)
at rubyjit.auto_create_table_E1FB7FE0CE1D294ABA67860B81492049B1D6AE7F.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb:71)
at rubyjit.auto_create_table_E1FB7FE0CE1D294ABA67860B81492049B1D6AE7F.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:87)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:78)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)
at rubyjit.method_missing_with_auto_migration_CA4124448766A2D0B44A8F3F5A4A9EC9908F2D02.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb:55)
at rubyjit.method_missing_with_auto_migration_CA4124448766A2D0B44A8F3F5A4A9EC9908F2D02.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:87)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:148)
at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:101)
at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:101)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:78)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)
at rubyjit.auto_create_table_E1FB7FE0CE1D294ABA67860B81492049B1D6AE7F.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb:71)
at rubyjit.auto_create_table_E1FB7FE0CE1D294ABA67860B81492049B1D6AE7F.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:87)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:78)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)
at rubyjit.method_missing_with_auto_migration_CA4124448766A2D0B44A8F3F5A4A9EC9908F2D02.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb:55)
at rubyjit.method_missing_with_auto_migration_CA4124448766A2D0B44A8F3F5A4A9EC9908F2D02.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:87)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:148)
at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:101)
at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:101)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:78)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)
at rubyjit.auto_create_table_E1FB7FE0CE1D294ABA67860B81492049B1D6AE7F.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb:71)
at rubyjit.auto_create_table_E1FB7FE0CE1D294ABA67860B81492049B1D6AE7F.__file__(/Users/mdesjardins/_work/sonymusic/src/checkout/vendor/plugins/auto_migrations/lib/auto_migrations.rb)
at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:87)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:78)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)

etc. etc. etc. ad infinitum. This was the “Aha!” moment. It turned out the problem was in my vendor/auto_migration plugin. What does this plugin do? I honestly have no idea – from the README, it looks like it’s a tool that applies schema deltas based on schema.rb instead of using migrations the intended way (hmm… that seems… misguided). As I said at the start of this post, I inherited this project. I wasn’t even using this thing anymore.

I deleted the plugin, and voila! My stack trace woes disappeared.

TL;DR: Use jstack if you’re stuck w/ a blown stack on JRuby. It works like a champ.

In a continuing effort to differentiate Skicast from the other ski condition apps out there in Android-land, I’ve released a new version of Skicast Pro that features some spiffy new homescreen widgets.  These widgets allow you to view the current conditions at your favorite resorts at-a-glance without opening Skicast. Tapping the widget opens the main app’s favorites screen.

Skicast’s sales have been, quite frankly, a bit disappointing. The data feed for the app is kinda pricey, and it doesn’t look like I’m going to recoup that expense.  I had assumed that there was a larger potential user base for Skicast than my other app, Tidecast. That may still be the case, but right now Tidecast downloads are outpacing Skicast Pro downloads almost 5-to-1.

So, to try to boost its flagging sales, I’ve added the widgets. I’ve also introduced House Ads in the Free edition that try to upsell the Pro edition. My AdMob ads only get about a 40% fill rate, so house ads seemed like an obvious place to try to plug the upgrade – this is especially true for getting the message out about the new features in the Pro version.

We’ll see how it goes! I intend to slow down the pace of Android development for a while to tackle a few other mobile platforms. I’ve started ports of both Skicast and Tidecast to iOS before, but I got distracted by other things and abandoned them. I think I might revive those projects for a bit … I’m anxious to see how they perform in the App Store vs. the Android Market.

Ceres Logic is proud to announce the follow-up to its first Android application, Tidecast, with Skicast.

Skicast screenshot Sunday River

Skicast is amobile application that allows users to get the latest information on ski resorts in real-time. Through a partnership with SnoCountry.com, Skicast is able to provide the following information on ski areas in North America, Europe, and other ski areas in the southern hemisphere:

  • Open Alpine and Nordic trails
  • Trail conditions
  • Night Grooming
  • Trail Maps
  • Web Cam Views
  • Contact Information and Driving Directions
  • Weather Forecasts and Recent Snowfall totals

Enhancements to geo-locate nearby resorts are already underway, as well as Blackberry and iPhone versions. You can get to the Google Market page for Skicast by following the QR Code below.

qrcode

I’m working on an Android project right now where I plan on using a WebView to display some content, and I need to generate that content dynamically based on the results of a web service request. I wanted an easy-to-use templating language to build the pages with. I’ve worked with both Velocity and Freemarker, and either would’ve been fine. I settled on Velocity because it was a bit easier to set up to work with Android. Here’s how I did it.

Setup Logging

First, I wanted to setup Velocity to use Android’s built-in logging system. To do that, I needed to create my own logging class that implemented the LogChute interface.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
package com.cereslogic.velocity;

import org.apache.velocity.runtime.RuntimeServices;
import org.apache.velocity.runtime.log.LogChute;

import android.util.Log;

public class VelocityLogger implements LogChute {
    private final static String tag = "Velocity";

    @Override
    public void init(RuntimeServices arg0) throws Exception {
    }

    @Override
    public boolean isLevelEnabled(int level) {
        return level &gt; LogChute.DEBUG_ID;
    }

    @Override
    public void log(int level, String msg) {
        switch(level) {
        case LogChute.DEBUG_ID:
            Log.d(tag,msg);
            break;
        case LogChute.ERROR_ID:
            Log.e(tag,msg);
            break;
        case LogChute.INFO_ID:
            Log.i(tag,msg);
            break;
        case LogChute.TRACE_ID:
            Log.d(tag,msg);
            break;
        case LogChute.WARN_ID:
            Log.w(tag,msg);
       }
    }

    @Override
    public void log(int level, String msg, Throwable t) {
        switch(level) {
        case LogChute.DEBUG_ID:
            Log.d(tag,msg,t);
            break;
        case LogChute.ERROR_ID:
            Log.e(tag,msg,t);
            break;
        case LogChute.INFO_ID:
            Log.i(tag,msg,t);
            break;
        case LogChute.TRACE_ID:
            Log.d(tag,msg,t);
            break;
        case LogChute.WARN_ID:
            Log.w(tag,msg,t);
        }
    }

}

You can obviously adjust the isLevelEnabled method for your desired logging level.

Create a ResourceLoader

Next I need to feed my templates to Velocity. I could have read my templates manually as files from the assets directory, then passed the contents of the templates file to Velocity.evaluate as a String. But Velocity has a very configurable way to process templates that, enables it to cache templates internally, so I decided to try that.

When passing Velocity the name of a template file, it delegates the template loading to a ResourceLoader class. When you initialize Velocity, you can configure which ResourceLoaders it should use to find and read your templates.  Later. when you call the getTemplate method of the Velocity helper class, you pass it the name of the template that you’d like to load as a String. Velocity will pass the resource name down to its ResourceLoader(s).

I wanted to store my Velocity templates in the raw subdirectory of the res directory in the Android project, so I needed to build a ResourceLoader that could do that. I decided to extend Velocity’s built-in FileResourceLoader as a starting point. Here’s what I came up with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
package com.cereslogic.velocity;

import java.io.InputStream;

import org.apache.commons.collections.ExtendedProperties;
import org.apache.velocity.runtime.RuntimeServices;
import org.apache.velocity.runtime.resource.Resource;
import org.apache.velocity.runtime.resource.loader.FileResourceLoader;

import android.content.res.Resources;

public class AndroidResourceLoader extends FileResourceLoader {
    private Resources resources;
    private String packageName;

    public void commonInit(RuntimeServices rs, ExtendedProperties configuration) {
        super.commonInit(rs,configuration);
        this.resources = (Resources)rs.getProperty("android.content.res.Resources");
        this.packageName = (String)rs.getProperty("packageName");
    }

    public long getLastModified(Resource resource) {
        return ;
    }

    public InputStream getResourceStream(String templateName) {
        int id = resources.getIdentifier(templateName, "raw", this.packageName);
        return resources.openRawResource(id);
    }

    public boolean  isSourceModified(Resource resource) {
        return false;
    }

    public boolean  resourceExists(String templateName) {
        return resources.getIdentifier(templateName, "raw", this.packageName) != ;
    }
}

Because the templates are statically bundled with the .apk file, we can assume that Velocity’s caches don’t need to concern themselves with modification times on the templates, which is why getLastModified and isSourceModified don’t really do anything.  The getResourceStream and resourceExists methods lookup the resource ID by name. The commonInit method is called when the ResourceManager initializes the ResourceLoader. You’ll notice that this is where we stash the package name for the resources as well as an instance of the Resource class.

Use It

So to use what we just created, we need to do some configuration before we call Velocity.init(), which will look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
public class MyActivity extends Activity {
  private void setupVelocity() throws Exception {
        Velocity.setProperty(Velocity.RUNTIME_LOG_LOGSYSTEM_CLASS, "com.cereslogic.velocity.VelocityLogger");
        Velocity.setProperty("resource.loader", "android");
        Velocity.setProperty("android.resource.loader.class", "com.cereslogic.velocity.AndroidResourceLoader");
        Velocity.setProperty("android.content.res.Resources",getResources());
        Velocity.setProperty("packageName", "com.cereslogic.myapplication");
        Velocity.init();
  }
.
.
.
//
// Somewhere where we want to use velocity:
//
    WebView engine = (WebView) findViewById(R.id.web_engine);
    Template template = null;
    try {
        setupVelocity();
        VelocityContext context = new VelocityContext();
        // add stuff to your context.
        template = Velocity.getTemplate("mytemplate");
        StringWriter sw = new StringWriter();
        template.merge(context, sw);
        engine.loadData(sw.toString(), "text/html", "UTF-8");
    } catch (Exception e) {
        // deal with it.
    }

In the setupVelocity method, we need to configure Velocity to use our new ResourceLoader and Logging classes, and configure the package name for our resources, just before calling Velocity init. Note that, if you name your template mytemplate.vm, you’ll only pass mytemplate to the Velocity getTemplate method. This is because of the idiosyncratic way that Android’s named resource lookup stuff works.

Now you’re ready to use Velocity in your Android project!