I’ve been thinking how much things have changed lately at Ignighter.  We’re starting to get some press, a bunch of daily registrations, and a bunch more messages piping through our once-rinky dating website.  It’s beginning to feel like we’re not such a small startup anymore.

When milestones like those pass, it really changes the way your team builds software.  These days, I’m obsessed with building at scale.  Back when we founded Ignighter back in 2008, we built a system that was as efficient and scalable as an old rusty tricycle.  For the past 3 years, we’ve re-engineered (and re-engineered, and re-engineered) the system, and these days it feels like we’re building a jetliner right just as it’s taking off.

It’s a lot of fun.

Anyway, I wanted to share a new project I’m working on to help me keep an eye on everything as we fire up the afterburners.  This week, we launched the Ignighter Early Warning System.  It’s a private twitter account that keeps an eye on our logs and let’s me (and my team) know when there is significant movement, up or down, in them.  I’ve configured it to tweet out changes in our error, stats, info, or debug logs.  And since it’s entirely homebrew, it’s completely customizable.

These metrics are for illustration purpose only; They do not represent actual Ignighter.com usage.

Since our developers are all already on it, it makes sense to share this information on twitter.  While this system supplements, not replaces, traditional regression and Unit Testing, this project allows us to act on production issues before they snowball into a full-blown catastrophe.

Time from hallucination to first iteration: 3 hours.  If you’re interested in building one for your project: here are the tools I used:

• Syslog-NG to aggregate our logs
• Plain vanilla bash scripting, scheduled via a crontab
• TTYTwitter to tweet updates
• And, of course, twitter

Next up, I’m looking to extend the project to text us when a particularly egregious swing our statistics occurs.

If you build your own, I’d love to hear about it!  Leave me a tweet or a comment below.

Did you know? Ignighter is hiring.  We’re based in NYC, work hard, have a lot of fun, build cool shit, and we’re backed by some of the best investors in the business . Check out our open development positions.

 

Note: Any information of proprietary value to my employer has been removed or approved, and this post has been approved by my employer.