The plan is to be able to use computer vision to detect objects or events, send events based on detection, process the event, then send an event or command back down to the device to perform an action. We’ll start with local development then explore cloud in future posts. Yup, the end-to-end analytics platform project is growing in scope, that’s okay.
Lights, camera, action! 🎥
Now to detect, capture, and even track objects with a computer vision solution we need something that can ‘see’ and ‘move’. The Pimoroni has a pan-tilt hat to ‘move’ with servos to pan (x-axis) and tilt (y-axis) the mounted camera. We also have a NoIR Pi Camera to help ‘see’. NoIR means NoInfrared. Why not the normal one? This one basically has night vision. Case closed.
Compute, uh, vision? 😁
After removing the SenseHAT we used before, which we can see in the background. We can use the Pimoroni guide for assembly to get the new hat set up. One thing we don’t have is the Neopixel stick (light) which we don’t need.
Interesting point 💡
Pan-Tilt HAT is a two-channel servo driver designed to control a tiny servo-powered Pan/Tilt assembly. - Pimoroni pan-tilt hat Github repoWhat's a servo? A servomotor (or servo motor) is a rotary actuator or linear actuator that allows for precise control of angular or linear position, velocity and acceleration.[1] It consists of a suitable motor coupled to a sensor for position feedback. It also requires a relatively sophisticated controller, often a dedicated module designed specifically for use with servomotors - Wikipedia
We’re focused on getting the pan-tilt working, not the camera. We’ll set up the camera when we do the object detection, image capture, etc. The kind maintainers of the Pimoroni pan-tilt hat Github repo have graciously bestowed upon us a curl command to install everything we need.
curl https://get.pimoroni.com/pantilthat | bash
Wait… “may explode”? 💥
Because we already enabled the I2C using the raspi-config tool, we can see the setup noticed that and printed out ‘I2C Already Enabled’.
Fully prepared 👍
We’re going to opt for the full install so that we can grab the examples and docs for future. You know, just in case 😏.
Joyful exclamation ❗
Installation done, let’s turn the key 🗝️ and see if this beauty starts.
Start up python in the VSCode bash terminal. Import the pantilthat library. The documentation has a few methods we can try out. We’ll start simple using the pan() and tilt() methods passing in the angles within the allowed range.
One hop this time 🕺
Great! It works. Playing around a little we can see the angle changing.
Hi WALL-E! 🤖
Queue music for interpretive machine dancing through numerous function calls… 🤣
All done! Before we close out, the documentation suggests it’s a good idea to turn of the servo drive signal to save power if we don’t it to actively point somewhere using the pantilthat.servo_enable(index, state) function. Reset the servos to their original position, and used the function to disabled the two servos. Now to shutdown the Pi and think about the next challenge. Getting the camera feed working, then on to object detection and tracking. I’d like to see if we can get the solution working with a Python venv though.
sudo halt
For fun 🎈
While perusing the documentation I noticed a note on displaying the Pi in the bash terminal which I thought was nifty. Give it a try…
pinout
Whaaaat! 🤯 Pi in bash!!
Summary
It’s been a quick post. Wrapping it up, we got our new pan-tilt hat installed, working, and dancing which brought a smile to my face. There’s heaps still to learn, I2C (I2C protocol), pinouts, and so much more. Most of which is new to me too.
Quick note 📝
A massive thank you to the many people who put their time and effort into projects, like the Pimoroni pan-tilt hat repo, which make things significantly easier for all of us.
We’ll work on the vision part in the next post. Camera’s, image capture, and even object detection. Once we have that done, we’re going to start working on connecting the device to the cloud.
TLDR; After a suggestion from a friend (Waylon Payne LinkedIn) we start learning about time series database InfluxDB. We set up InfluxDB on the Raspberry Pi by creating a database. Work on getting Grafana installed and running. We write, troubleshoot, and learn a bunch logging data to InfluxDB. Finally we create a dashboard in Grafana to display on Sense HAT telemetry.
Begin the Influx 🌌
Last time we tackled writing out SenseHAT readings to a csv on the Pi. Now though, we level up by working on writing that data to a database more suited for streaming log data, in this case InfluxDB.
A few things here to learn from the previous code snippets:
apt – Is a command line package/software management tool on Debian (Debian Wiki) like search, installation, and removal.
etc directory – Holds core configuration files. Found a nice Linux directory structure for beginners post.
wget – Is a command line package/software for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols (Debian Wiki).
tee – Reads the standard input and writes it to both the standard output and one or more files (GeeksForGeeks).
source – Reads and execute the content of a file (GeeksForGeeks).
Best I can understand at the moment is that we get and store a public key that allows us to authenticate/validate the InfluxDB package when we download it by echoing the latest InfluxData stable release package and adding the package deb record to the source.list directory in a new file which seem to allow apt-get to pick up future updates. That kicked off the install of InfluxDB. How do we know? The console says so..
The influx begins!
It installs InfluxDB version 1.8.9 (not 2.0 which is the latest at the moment). Keep that in mind when working with documentation. Upgrading to 2.0 we can leave for the future. Onward!
systemctl – Is a command line utility to interact with systemd. It covers way more, but what I mostly used it for was working with the services.
Found the command to check a service status with the –help switch for the systemctl command. These all feel reasonably familiar coming from working a little with PowerShell and the Windows terminal.
sudo systemctl --help
sudo systemctl status influxdb.service
Running, running, running.
The service is up, active, and running. That means we should be able to connect to it. We can do that by logging into the Influx CLI from the terminal. Then creating a database. Creating a user. Finally, granting the user permissions.
influx
create '<yourdatabase>'
use '<yourdatabase>'
create user '<yourusername>' with password '<yourpassword>' with all privileges
grant all privileges on '<yourdatabase>' to '<yourusername>'
Ah familiar territory! A database! Now we have:
A database service running.
A database created.
A user that has more than enough permissions to interact with the database.
Grafana
We want a way to visualise the telemetry that’s going to be written into the database. Grafana gives us the ability to create, explore and share all of your data through beautiful, flexible dashboards and we can run the service on the Pi. We’re taking the same approach as we did for InfluxDB to get Grafana up and going. Getting all the packages, installing them, running updates, and validating the services.
Once the service is up we should be able to connect to it on port 3000.
Yay! We’re connected. Let’s log in with the user name and password ‘admin‘ then reset the password. After the login process is done, we’ll land on the homepage for our Grafana instance running on the Pi. I’m actually excited about this 😄.
We need a way to connect Grafana to our InfluxDB database. On the Home page is a ‘Data Sources‘ tile which we can follow to add a data source.
Source of the action 💥
We can use the search box to lookup a connector for InfluxDB. Once we have that we just select it.
Search the unknown 🔮
From there we configure the settings for the connector.
The way through the mountains 🏔️
Credentials to make our way into the database.
You shall pass 🧙♂️
Finally, we save and test the connector to make sure its all working.
Green.. Green is good 🟢
Good news. The scaffolding is in place. Now we need to get data into the database then configure some dashboards.
Bilbo Loggings 🪵
“If ever you are passing my way,” said Bilbo, “don’t wait to knock! Tea is at four; but any of you are welcome at any time!”
– Bilbo Baggins
Did someone say tea? Time for a spot of IoT! Now to get our IoT device capturing data. A quick swish of our telemetry logging code and we have a starting point. All we need to do is figure out how to log data to InfluxDB not the csv.
Our virtual environment. It’s so smart. I think it reads my blog drafts 🤣. After we chose the interpreter, VS Code switches context to our virtual environment context. It even reminds us in the status bar at the bottom of the window. So thoughtful!
Our friendship will continue.
🚧 Slight detour 🚧
Slight digression from our regular blog flow...
Now, if you are briskly following along and haven’t switched to the venv in the terminal, then you will probably run into an error like this. Which might lead you on a wild goose chase across fields of learnings and wonder.
Goose chase begins 🦢
Naturally, we go looking to see if we find any packages for influxdb-client:
pip search influxdb-client
Which the PyPI XMLRPC API politely, in a crimson message, lets us know things are not as peachy as we hoped:
⚠️!Unmanageable load!⚠️
Fault: <Fault -32500: “RuntimeError: PyPI’s XMLRPC API is currently disabled due to unmanageable load and will be deprecated in the near future. See https://status.python.org/ for more information.”>
After updating the pip version in the venv, we check on the status which the error message suggested. After a very interesting read, a smidge of despair, hope emerges… I said to myself, “Self, why does that terminal not have the venv prefix?“. That’s when I realised the true source of the problem. Me. I forgot to activate the venv 😅.
For the terminal we still need to activate the virtual environment. To do this on the Pi we can run:
source <yourvenv>/bin/activate
Activates PEBKAC fix to end goose chase 🦢
Behold!!! It lives!!
When we do that, the terminal actually changes a little, giving us a visual cue that we are in a Python virtual environment. Now to get supporting packages installed so that we can write Python code for InfluxDB. Taking a look at the InfluxDB Client Python GitHub Repo or the influxdb-client PyPI project.
pip install influxdb-client
Goose = chased! 🦢
Sometimes Python can’t catch the programmer being the error.
~ me
🚧 Slight detourends 🚧
The packages are installed. The logging can begin. To get started we need to import the influxdb_client into our project.
from sense_hat import SenseHat
from datetime import datetime
from influxdb_client import InfluxDBClient, Point
Yet again, we find a pebble in our shoe… While trying to import the sense_hat library modules in the REPL an error presented itself which seemed related to the way the numpy library was installed.
numpy dumpty had a great fall 🤣
The error message helps a ton! Jumping over to the common reasons and troubleshooting tips helps with options to solve the issue on a Raspberry Pi, if we use our original error “libf77blas.so.3: cannot open shared object file: No such file or directory“. I opted for the first option to install the package with apt-get:
sudo apt-get install libatlas-base-dev
Then tried to enter the python REPL again (just typing “python” while in the terminal) and importing the SenseHat module.
No one by that name here..
I am beginning to feel like a module hunter 🏹. I tracked down a RaspberryPi thread which led me to a comment on a GitHub issue for the RTIMU module error. To be clear, this doesn’t seem to be an issue when I am running in the global Python scope. Only an issue in the virtual environment. The folks were kind enough to provide a way to install this with a pip command. Here we go:
pip install rtimulib
Sparks joy ✨
Yes! It works!
I tried initially to write a Python function that would write to and query the database. It wasn’t long before I ran into an error trying to connect to the database using the Python function.
The settings are changed. To bring them into effect we need to restart the service.
sudo systemctl restart influxdb.service
Starting the engine
No much to go on. I found what seems to be a potential workaround. There is a comment further down in this thread on InfluxDB not starting that talks about adjusting a sleep setting for a start up file. Worth a try. Using nano again, we open the file and make the change.
Running, running, running 🏃
Time to write the code that will log records to our database. The idea is simple. Run a loop. Every few seconds get a Sense HAT reading. Log the reading to our InfluxDB. Stop the loop when we interrupt the program.
from sense_hat import SenseHat
from datetime import datetime
from influxdb_client import InfluxDBClient, Point
timestamp = datetime.now()
delay = 15
sense = SenseHat()
host = "localhost"
port = 8086
username = "grafanabaggins"
password = "<NotMyPrecious>"
database = 'shire'
retention_policy = 'autogen'
bucket = f'{database}/{retention_policy}'
def get_sense_reading():
sense_reading = []
sense_reading.append(datetime.now())
sense_reading.append(sense.get_temperature())
sense_reading.append(sense.get_pressure())
sense_reading.append(sense.get_humidity())
return sense_reading
# This method will log a sense hat reading into influxdb
def log_reading_to_influxdb(data, timestamp):
point = ([Point("reading").tag("temperature", data[1]).field("device", "raspberrypi").time(timestamp), Point("reading").tag("pressure", data[2]).field("device", "raspberrypi").time(timestamp), Point("reading").tag("humidity", data[3]).field("device", "raspberrypi").time(timestamp)])
client = InfluxDBClient(url="http://localhost:8086", token=f"{username}:{password}", org="-")
write_client = client.write_api()
write_client.write(bucket=bucket, record=point)
# Run and get a reading Forrest
def run_forrest(timestamp):
try:
data = get_sense_reading()
log_reading_to_influxdb(data, timestamp)
while True:
data = get_sense_reading()
difference = data[0] - timestamp
if difference.seconds > delay:
log_reading_to_influxdb(data, timestamp)
sense.show_message("OK")
timestamp = datetime.now()
except KeyboardInterrupt:
print("Stopped by keyboard interrupt [CTRL+C].")
I struggled for a while trying to figure out the bucket/token variable to what I was able to do in the 1.8.9 CLI easily. I revisited the Python client library and noticed a specific callout for v1.8 API compatibility which has an example that helped me define the token. It wasn’t long before we got the script running and data was being logged to the database.
We’re getting there!
To the Shire
Before we get started on logging data to the database we need to understand some key concepts in InfluxDB. It won’t be the last time I visit that page, these concepts are foreign to me. I learnt to use InfluxQL which is a SQL language to work with the data. There are some differences between Flux and InfluxQL that you might want to keep in mind. I had a tricky time figuring out how to execute Flux queries initially after I wasn’t getting any data back from my Flux commands in a Python function, but saw that you could invoke a REPL to test queries with). To keep things simple though, I opted for InfluxQL. We can launch the Influx CLI from the terminal and query our data.
influx
SHOW DATABASES
USE <database>
SELECT * FROM <table>
Successfully captured! 🪤
Let’s see if we can build a dashboard to visualise the data we are logging. We can connect to our Grafana server again. Head to the home page. There is a “Explore” menu item that is a quick way for us to query our data and experiment. Once the window opens up we select our data source connection from the drop down box and begin building a query with a wonderfully simple interface.
Visual query building 🤩
It’s at this point we realise that our logging design might not be correct. What I was expecting was that I could use the columns in the SELECT and WHERE clauses. Apparently not. I initially thought that design would work better because I understood that tags were indexed, not fields, so querying the tags would be faster. Good in theory, but I couldn’t reference the tag in the SELECT and WHERE clauses. My initial mental model needed tweaking. A change to the logging function to log to a single point, not three, with multiple fields.
point = ([Point("reading").tag("device","raspberrypi").field("temperature", data[1]).field("pressure", data[2]).field("humidity", data[3]).time(timestamp)])
Minor InfluxDB management needed in future to clean up the database. For now though, we have our ‘frodologgins‘ database which is empty. I ran the logging function against the new database and…
* Chef’s kiss *
It works as expected! A quick updated to the Grafana connection settings to switch to the new database. With the updates in place we now get the expected results in the drop down. We can see the fields we want to display and chart.
We can try reconcile the point, tags, and field in the Python code to how we are querying it with InfluxQL. Slowly sharpening our mental model and skills. The query reads as follows:
From our database
Query our readings for the default/autogen retention policy
Where the device tag value is raspberrypi
Return the last temperature field reading
Group by ten second intervals
One thing I wasn’t quite sure of is the way that the time range worked in Grafana with the data logged in the database. The query looked correct but no data was returned. I was looking at a window from now-1d to now initially. It seemed logical to me, “find me all the data points from yesterday to now“. The Inspector in Grafana helps get the query and then we can use that to run the query in the Influx CLI to test the queries.
Inspector Clouseau 🕵️
I eventually adjusted that to now to now+1d which in my mind is “back to the future” 🔮🚗, but it worked. I think this comes down to how the dates are stored (i.e. timezone offsets) and the functions evaluation. I’ll dig into that later, for now this works, we have data showing on a graph.
Graph
Let’s take the learnings and apply it to building the dashboard. Head to the home page. There is a “Dashboards” tile we can use to build our first dashboard.
Dash lightning! 🌩️
It opens up a new editing window. I chose an empty panel. From there we can edit the panel in a similar way to what we did with the Explore window. In the upper right corner we can choose the type of chart.
Serious time ⏳
There are a bunch of options from changing the charts, adjusting threshold values for the gauges, applying units of measure, and so much more. For our case that’s “Time series“.
That’s it! Use the same approach to build out the other charts. I added “Gauge” visuals as well with the corresponding query.
Ice, Ice, Baby 🧊
Learnings 🏫
We made it! It took a while but we did it. Failure is a pretty good teacher. I failed a bunch and learnt a more. That’s not wasted time. It’s worth just getting hands on and trying different things out to build the mental model and skills. I have a long way to go to really understand Python, InfluxDB, Grafana, and Linux but I’ve made progress and learnt new things which is a blessing.
If you are wondering how I am writing code remotely on the Pi, go check out setting up remote development on the Raspberry Pi using VS Code. We are using the same approach here to get connected and working on our Pi.
Part of this journey is growing my skills. I chose Python as a programming language. Not diving into too many details. It just gives a range of capabilities (web through to machine learning) with a single language. No need to switch too much while learning all the techs in this series. Works for me.
While upping my Python game I came across something called virtual environments. A little primer on virtual environments. I think I have a reasonable grasp on how to start using them for better package management.
Not going full virtualenvwrapper yet though. “Hey, I just met you and this is crazy, but here’s my bookmark, browse it later maybe. #justsayin’
To that note, let’s set up a virtual environment. First, check our Python versions on the Pi:
python3 --version
Snake in eagle shadow 🐍🦅
We have Python3 installed on the Pi. That means we should have the venv capability built-in. Let’s give it a whirl!
python3 -m venv noobenv
Environment cultivated
When we do that a new folder gets created in our repo. It has a bunch of folders and files related to the “inner workings” of how virtual environments work <- science 👩🔬.
Logging the things 🪵
The goal here is that we have an IoT device that is capturing data from the sensors. It has a bunch of sensors we are going to use, which is exciting. Honestly, the more I work with it, the more amazing it is to me.
from sense_hat import SenseHat
from datetime import datetime
sense = SenseHat()
def get_sense_reading():
sense_reading = []
sense_reading.append(datetime.now())
sense_reading.append(sense.get_temperature())
sense_reading.append(sense.get_humidity())
sense_reading.append(sense.get_pressure())
sense_reading.append(sense.get_orientation())
sense_reading.append(sense.get_compass_raw())
sense_reading.append(sense.get_accelerometer_raw())
sense_reading.append(sense.get_gyroscope_raw())
return sense_reading
We create a function (get_sense_reading) that we can call repeatedly. Then use the SenseHat functions (e.g. get_temperature) to get readings from the different sensors. To get them all in a single object/row, we can use a list (sense_reading). Then append each reading to the list. Once we have them, we return the list object.
Witness the quickness ⚡
We add a for loop to our code to call the get_sense_readingfunction a few times are print the results to the terminal window. We can run the program (main.py) by calling the Python 3 interpreter and passing the file name to it. That loads the code, executes the loop, prints the results.
python3 main.py
Now to add data to a file on the device. We’ll use a CSV for now, then adapt it later based on our needs. We can use the the sense_readingobject returned by the get_sense_readingfunction and write that to the file using the csv library.
from csv import writer
timestamp = datetime.now()
delay = 1
with open("logz.csv", "w", newline="") as f:
data_writer = writer(f)
data_writer.writerow(['datetime','temp','pres','hum',
'yaw','pitch','roll',
'mag_x','mag_y','mag_z',
'acc_x','acc_y','acc_z',
'gyro_x', 'gyro_y', 'gyro_z'])
while True:
data = get_sense_reading()
difference = data[0] - timestamp
if difference.seconds > delay:
data_writer.writerow(data)
timestamp = datetime.now()
We start with a timestamp because we want to calculate a delay interval, say 1 second, between writes to the file. The open the file and write a header row (writerow) to the file. We use a while loop to collect readings, then once we exceed the delay interval, write the row to the file. We need to update the timestamp otherwise we will write a row on every pass after we exceed the timestamp the first time.
Testing seems to be working and we can log data to a file on the device. VS Code integrated terminal really is fantastic at running multiple and side by side shell/terminal windows.
A tail of two terminals 💻
Awesome! It works. We have a program logging data to a csv file at a defined interval. Tail simply prints end of file content. A few lines is all we need to double check things are working. Last thing left.. shut down the Pi remotely. Usually I would use a shutdown command. I gave a new command a try “Halt”.
sudo halt
Halt! Who goes there? 🛑
Looks like that worked 🙂 The connection got terminated and VS Code detects that, and tries to reconnect. Pretty slick. We managed to start putting new Python skills to use. Learnt how to create a virtual environment for better package management. Then collecting and writing telemetry from the SenseHat to local storage on the Pi.
TLDR; My very first Pi and sense HAT was graciously gifted to me by Jonathan Wade (LinkedIn). I assembled it. Tried to go headless. Ended up adding a head because reasons. Ran through initial setup and updates. Configured OpenSSH on Windows and SSH on the Pi to get to the headless state.
I got gifted something amazing! Yup, my first Raspberry Pi. Not only that, a Sense HAT too! Now for many people that might mean much, but this is a pretty big moment for me. To save you from another unboxing experience I took the liberty by doing that privately and cut to the end. Behold! The unboxed product:
Bare metal
Looking through what we have here. The Raspberry Pi 4 Model B (top left), the Raspberry Pi Sense HAT (bottom middle), a SanDisk 32GB microSD card, some spacers, screws, power cable, and a HDMI to mini HDMI cable.
Fresh off the factory floor
Assembling the unit was really simple. Just a quick look at the Sense Hat board and we can see some amazing things:
Air Pressure sensor
Temperature and humidity sensor
Accelerometer, gyroscope, and magnetometer
8×8 LED matrix display
Even a small joystick!
This device is pretty EPIC and it’s not even powered it on yet. So many things I haven’t ever worked with but can’t wait to try and figure them out.
Like a diamond 💎
Next up power and networking. The moment I connected the power a rainbow 🌈 filled the room. A sign of a pot o’learnings 🪙 to be found at the end of this experience.
Nice! Now we have the whole unit assembled. What’s the plan? Well, the thinking is to use this to deploy and run Azure SQL Edge on it. Why? A few reasons:
I have never worked with a Raspberry Pi
I haven’t really work on Linux at all
I have never done any work with Azure IoT solutions, or IoT at all for that matter
I do know Azure SQL reasonably well, though not Azure SQL Edge
That didn’t work. Apparently SSH has been disabled by default. Considering I don’t have a microSD card reader, it’s time to put a “head” on nearly headless Pi and connect a screen 🖥️. The HDMI cable, a keyboard, and a mouse later and we are connected. I ran through the setup, updated the password, downloaded the latest updates, then set up SSH. There are other security best practices that I am going to follow as well after this post. Then tried to connect again and…success!
Connection suck seeds! 🌱
Next I shut the Pi down. Disconnected the screen, mouse, and keyboard. I’m going to try work on this device remotely so I don’t need those peripherals right now.
Now that we have an IoT device I am going to start exploring if there are any open data sets that I can start using and feed some of the device telemetry into the end-to-end analytics solution as a cohesive project. We are going to set up additional services in our solution to support IoT device which will be fun.
No, I am not talking about luxurious battle station set up with StreamDecks, DSLR cameras, lighting, RGB keyboards, etc. Someday, maybe, when I am smart enough to figure out all the audio visual stuff. What I am talking about though is the setup for my development on my machine.
Couple of things I am working with:
Visual Studio Code Insiders Edition
Visual Studio 2019 Enterprise Edition
Azure Data Studio
SQL Server Management Studio
Docker Desktop (running under WSL2, not that I needed WSL2, I just thought it might be worth trying out)
Windows Terminal
Git for Windows
Visual Studio Code
The not so new kid on the block. I find the extensions really good. There seems to be a bunch of investment in this tool. It covers a really wide range of uses for what I do. Download it here: Visual Studio Code
Visual Studio 2019
Look I haven’t used this in a while. If I actively start working with it again, I will loop back on this. I generally don’t do much customisation with Visual Studio. I do generally make sure that I have it geared for Azure, Database Projects, and either C# or Python development. I find it does most of what I need to do out the box.
Azure Data Studio
The data sister of Visual Studio Code. Makes sense to have. Get started here: Azure Data Studio
SQL Server Management Studio
Ye old SSMS. This has been my world for the past few years as a SQL Server DBA and Consultant. Download it over here: SQL Server Management Studio
Docker Desktop
Containers are a hot topic. An area that I am looking to explore a little more. Thankfully, the team at Docker make it really easy to get up an running on my Windows machine. Get started over here: Docker Desktop
Windows Terminal
I love this thing. I was not a “command line guy”. The more I work in The Cloud, the more I find myself enjoying it. Considering I spend more time there, why not make it pretty 🦋. At this point I welcomed Scott Hanselman’s Pretty Prompt post, go check it out. Get started here: Windows Terminal
Git for Windows
Git for Windows. VSCode does have Git integrated, I have this for some other reasons.
That’s pretty much if for now. I used to use Notepad++ but I found that I can do most of what I wanted with VSCode.
Anthony Fourie 