Goal: Buy groceries. In simple caveman’s terms: “Get food”.
Problem: You know, the virus. I don’t want to go grocery shopping and wait in line to get into the store.
Solution: Order groceries online.
Another problem: The nearest delivery day is more than 1.5 months away. Not good.
Possible solution: Sit and refresh the page for hours, hoping that some slots free up and get back into the pool. Then, hope to be quick enough to snatch one of the slots.
Better solution: Remember that you can do anything in Python. Remember that I saw some useful stuff in an aptly titled book recently: “Automate the Boring Stuff with Python”. Go through the book, quickly learn to control the browser, write a program that checks everything automatically and reserves the nearest delivery slot as soon as it becomes available.
Details:
It was a few weeks ago. When I was failing at my main mission to bring food to the table, I came up with the idea above. I started working on it in the morning. I had to start pretty far back, going through the book and learning the first steps to control the browser with Python.
It was April 17th and the nearest delivery slot was for June 4th.
At first, I wanted to do something simple, similar to what Utku described in his article I had to build a web scraper to buy groceries. But in my case, it wasn’t possible. Setting session cookies and firing GET requests didn’t work. It had to be the whole shebang: with the browser, logging in, entering all data and simulating all the user actions.
I had the first working version around 2 AM and was about to go to sleep but then I decided that I need to automate it and make it check the availability every minute, while I would be sleeping.
I had two criteria to decide if the program should claim the slot automatically: it has to be earlier than June and the time of delivery has to be later than 12 PM. I didn’t want to be woken up at 6 or 7 AM. With that finally automated, I went to sleep around 6 AM, having coded since the morning of the previous day.
While I was sleeping, around 9 AM, the program found and quickly claimed the slot that opened up. But it was for May 23rd, more than a month away. It wasn’t good enough. When I checked the results in the morning, I canceled the reservation and modified the date to be in April. May was still too far away. Meanwhile, the nearest free slot was again for June.
15 minutes later, while I still didn’t get a chance to get dressed, the program did what it was supposed to do: it found and booked the new slot. For two days later! For a fitting time: 12:30 PM.
The moment of triumph:
A quick summary of what the program did:
Technical details:
MacOS, Python 3.7.3, Selenium, crontab.
Things to improve for the future, if I were to use it more than once:
Aftermath:
I shared a quick write-up on my Facebook profile, thinking that it could be interesting or funny for a few people.
Instead, it became my most-liked, most-shared (400+ shares in total), most-commented post, beating everything else by an order of magnitude.
And I was able to do all this from scratch, while not even being a programmer! I was sure programmers were automating such stuff left and right. Turns out, very few of them did.
The grocery chain’s representatives even got in touch with me, congratulated the inventiveness and saying “By the way, the board of directors says hi.” That was certainly unexpected :)
In Facebook comments, the grocery chain’s representative said they do free some slots for the near dates, at some specific hours. They were sure people knew that but nobody did.
A week or two later, when I visited the website again, I saw the information about the times when they free up slots plastered all over the site: as a huge pop-up on entry, as an info bar at the top, as a notification in their app and they even sent a broadcast email to all their customers with this info.
↑ Like what I'm writing about? Follow me on Twitter.
↓ or get updates when I publish a new article, review a new book or share a new #TIL.
Tech, growth, books, data science, more.
‘Today I Learned’. As random as it gets. But interesting.
50+ books per year can’t go to waste. Short summaries and recommendations.
Who am I? And why is the shark velvet?
Conversion optimization and growth consulting. It’s how I make my living.