Retrieving Random Records in Rails

What is the best way to retrieve one or more random records in Rails?

Given that there are multiple ways to retrieve random records in Rails and Ruby (take a look at this SO post), it can be confusing and frustrating to decide which course of action to take.

“Is there a technique to retrieve random records that is correct/general/Rails way/cross-database compatible? “

In this post, I’ll cover a couple of simple approaches that you can take to figure out what the best way might be for you.

The first priority that you likely have to keep in mind is speed. Most applications have a threshold for what is considered slow – in web apps for example, anything that takes longer than a few hundred milliseconds to load is considered slow. So you don’t want to be choosing a method that will have your users waiting around forever for your app to do something. That being said, there is also no point in sacrificing code readability, ease of re-use and just time in general, to get faster than you have to.

The second thing you have to consider is the size (number of rows) of the database table you are looking to retrieve a random record from. In general, the bigger your table, the more knowledgable you’ll have to be about your database platform (typically Postgres or MySQL) to get to your desired speed.

Take this common solution to retrieve a random record in Rails:  

random_user = User.all.sample

To use the code above requires no internal knowledge of the database you’re running on. The #all method is a well-known ActiveRecord method and #sample is a Ruby method on Array that retrieves a random value from a given array.

The flip side of this approach is that if your database has a sizable number of rows, it will take a long time to execute. Try it out on your Rails console and see! The reason it does so is because User.all retrieves all the users from the database, and then builds ActiveRecord objects from them. So if you have let’s say 10,000 users, building 10,000 ActiveRecord objects takes a while.  If you’re reasonably certain that the table in question will not grow much in size over the lifetime of your app, and the speed at which the above code executes is agreeable to you, then by all means go for it.

Now, if your table is HUGE (has a million+ rows for example), the above method is not going to work for you. Consider using the ActiveRecord #offset and the Ruby rand methods. The #offset method in ActiveRecord specifies how many rows to skip before returning the rest of the rows. The rand method, as expected, generates a random number less than a maximum value – so when you say rand(10), it will generate a random number less than 10.

So something like this:

count = User.count
random_offset = rand(count)
random_user = User.offset(random_offset).first`

…works better (faster) for big tables, because you’re not making the database scan through every single row in the table.


Business Spotlight – Rachio

Business Spotlight brings into focus highly innovative companies in the home automation space and deep dives into what makes these companies tick. We’ll explore the technologies they use, the challenges they face and often successfully solve, and try to gain an insight into what it takes to build and execute a game changing product.
Meet Rachio, the makers of the Iro, a smart sprinkler controller. Founded by an experienced team of software designers and engineers in response to drought conditions in the western US, the Iro is the culmination of relentless research & development into:
  • Understanding the best and most effective ways to manage water in the context of outdoor irrigation systems.
  • Building and deploying a sophisticated cloud architecture that is able to bring to bear years of irrigation research to the consumer’s home and,
  • Does so in a user-friendly way, such that the user is not overwhelmed.

The outcome of this research is an irrigation controller that for the first time, is worthy of controlling access to one of your most precious resources. The Iro is not only easy to use with your smartphone app, but also knows exactly when and how much water to use depending on current and past weather conditions, the type of soil, the slope of your zone, the type of nozzle you have, the type of vegetation and many other variables, greatly increasing the efficiency of your irrigation system and saving costs substantially.

17466169758_d0244d98d1_o Why Rachio was founded Rachio was started in the summer of 2012, by founders Chris Klein and Franz Garsombke. Chris and Franz were in the middle of a serious drought situation in Denver, Colorado when, inspired by a state-wide conversation on tearing out lawns and replacing them with vegetable gardens, they decided to look into how big of issue outdoor irrigation really was. Chris found that outdoor irrigation contributes to a significant amount of water waste in the US (about 8 billion gallons a day), and that for homes with lawns in particular, 50% of water usage went towards outdoor irrigation. After some research on the subject, and being software experts, they ultimately settled on asking the question, “What can software do to solve this problem?” Exploration Phase The first thing they explored was using a wireless outlet to which a standard irrigation controller could be plugged into. The idea was that via WiFi, they could turn the wireless outlet on and off and subsequently control the irrigation schedule. However, preliminary prototyping efforts with a WeMo controller quickly tabled that idea. They found that the closed architecture of the WeMo did not lend itself well to a suitable solution. They then directed a couple of months worth of R&D effort into devising a solution that would not need any additional hardware, ie, something that would be purely software. In the end, they found there was no way around building hardware, so that’s what they did. Biggest Challenges According to Chris, Rachio’s biggest challenges during this early phase mainly concerned their entry into the hardware products space. Being a team of software experts building a hardware company, and the hardware business being a capital intensive one, they found that their initial lack of experience got in the way of raising funds. For example, to know how much funding they would need, they had to accurately model manufacturing costs considering the huge cost variability of various hardware components. Chris says that winning a state-wide competition for innovators and entrepreneurs was a turning point for Rachio. They not only won a substantial amount of prize money, but were also able to befriend a local manufacturer in the audience. Working out of their new friend’s plant they were able to build relationships that helped them successfully navigate the hardware space. All of Rachio’s manufacturing is based in Colorado, which Chris feels gives them a significant competitive advantage. Pitfalls avoided According to Chris, the obvious things like product-market fit, proof of concept, the relationships you build and the connections you make all play a role in ensuring success for your product and company to-be. Manufacturing in the US, with the manufacturing plant located about an hour away, allowed them to ramp up and down according to demand and remain fluid. Rachio also took a software first approach to build the product. Understanding that the hardware was essentially a shell and that any deficiencies in the software such as bad connections or inaccurate algorithms would be perceived by the consumer as a defective product was a key insight that helped Rachio maintain focus on building high quality software for the Iro. Rachio also benefited from Chris’ background in construction systems management. His experience in building hard goods, commercial buildings and structures became increasingly relevant as they began to manufacture the Iro. Familiarity with managing subcontractors, exposure to lead times and the ways dependencies work in the hard goods world (more of a Waterfall approach, as opposed to Agile) all helped Chris and his team at Rachio when the time came to successfully and consistently make progress. 17783811778_895f179e5a_o Tech Stack The main components of Iro’s brain are comprised of:
1. The internal firmware, which resides on board the Iro
2. A mobile app, which the user uses to configure the Iro
3. The cloud app, which lives on Rachio’s servers and contains the bulk of Iro’s intelligence and business logic.

Firmware The firmware that sits atop the Iro is lightweight in nature, and its main job is to maintain connection to the WiFi network and communicate back and forth with the Rachio cloud app. Communication to the Rachio app involves listening for instructions and reporting back on current status. The firmware also has the ability to continue irrigation and maintain a healthy yard if WiFi connection is lost. Rachio uses an Electric Imp Wifi chip to run the firmware, and according to Chris the firmware is written in Squirrel, which is a “Javascript-y” language provided by Electric Imp. The firmware engineering team consists of a full-time firmware engineer, and Franz, Rachio’s co-founder and CTO. Mobile App The mobile app holds a key position in the Rachio ecosystem, by virtue of being the primary point of contact between the Iro and the user. When Chris and his team first designed the app, what made sense at the time was to design it so that the user felt like they were interacting with a piece of hardware. Chris remembers that getting the design right for the app the first time around was very difficult, owing to the fact that the Iro was one of the very first irrigation controllers to offer full configuration to the user via a mobile app. Considering the usability constraints of an irrigation system, such as multiple zones that can’t run on top of each other, calendaring, different soil types, valves, vegetation types, irrigation systems that run on pumps, a lot of thought had to be put in to the design and user experience (UX) so that the user was able to realize the benefits of having the powerful Rachio cloud at their fingertips. As the years and months have gone by, Rachio have gradually expanded and extended the feature set of their mobile app based on user feedback. As they’ve started to understand the IoT space better, Chris and the Rachio team have undergone a shift in their approach to app design. They are realizing that although the way one interacts with hardware might initially influence how the app is designed, ultimately, what yields good design is considering how one wants to interact with the system as a whole. Actively trying to understand the important elements of the user interaction, be it informational or configurational, and determining the level of abstraction at which the user can harness the widest feature-set possible, in as enjoyable a manner as possible, is key. Version 2.0 of the Iro mobile app is on the way, and according to Chris, it will represent a significant change in how users interact with the Iro stack. The Iro Controller Cloud The Rachio cloud contains the bulk of the business logic and is the real brains behind the magic of the Iro. The Rachio cloud is a multi-layered system, and has a public API layer that allows for client interaction with the cloud. Rachio’s own clients, like their iOS and Android apps, use this public API. The public API is also accessible to any other device that wants to integrate with Rachio’s services and the Iro. Given the competitive nature of the space, it isn’t possible to divulge much about the internals of the Rachio cloud other than to say that it is built in a type-safe language. Rachio also uses Node servers that stand in front of the API layer. Irrigation Scheduling An important component of the Rachio cloud is the irrigation scheduling algorithm. The main function of the irrigation scheduling algorithm is to determine, based on weather and zone data, how frequently your lawn should be watered. When the Iro was being built, and Chris and the team at Rachio were looking into irrigation algorithms, they found that prior research results on the subject were tried and true and should be the way to water. The Iro supports two types of irrigation algorithms:
1. The Penman equation
2. Maximal allowable depletion The Rachio cloud also ties into weather services and personal weather stations to obtain data such as daily evapotranspiration rate which are important inputs to the chosen irrigation algorithm. The next version of the Iro app and cloud infrastructure will allow the user to choose which weather station to use.

18718204015_72d2636c89_o Conclusion Chris says that although all the individual pieces of the irrigation solution have been available for a long time (for example, Kansas provides a spreadsheet to help home owners manually figure out irrigation scheduling), Rachio, by building a sophisticated cloud architecture and apps that encourage the user to interact with the cloud, has successfully managed to tie all the pieces together to create a water efficient and user-friendly system.

Rachio is at an exciting point in their evolution. They have had the opportunity to see their system in action for the last few years with real people and homes, and now have a much clearer vision of how the space they are in will evolve. They are hard at work at realizing this vision and are confident that they will continue to provide value in the years to come. Author Notes: I’d like to thank Chris Klein, CEO at Rachio for being so generous with his time.