General Raspberry Pi Sites:
Headless RPi Related:
- Adafruit RPi B+ Headless via Ethernet Product Wishlist
- General Tutorials:
- Headless RPi Blink LED via Python and VNC Server-Viewer/XFCE Desktop/IDLE Python IDE Step-By-Step Setup aka RPi Blinky Program:
- Watch Mr. C.'s Video:Headless RPi Setup
- Do this: Headless RPi Setup Connect via SSH file using Putty - HackerNoon.com
- Supporting Information: How to Create a File With No Extension Using Notepad *you will need this*
- I configured my router to always give the RPi the same local IP number aka a "static IP address" so I just enter once in Putty. I saved this static IP (mine is 192.168.1.129) in a Putty "session" file and just "Load" it when I want to use Putty to login to the RPi. Later- I will login using VNC Viewer and a desktop GUI (XFCE).
- Initial Configuration Using "sudo raspi-config" at CLI via Putty:
- Read "raspi-config" at raspberrypi.org
- "sudo raspi-config" tool options:
- Change User Password Do this. WRITE it down!
- Network Options I chaged my HOSTNAME.
- Boot Options I started with CLI login, changed to GUI login after XFCE install.
- Localisation Options Added en_US.UTF-8 UTF-8 not sure what conflicts having GB (default, Great Britain) also selected will arise, changed timezone, etc.
- Interfacing Options Enabled SSH, VNC (uses 168 mb) I2C, and GPIO remote access.
- Overclock Configure overclocking for your Pi
- Advanced Options I expanded filesystem. Set resolution to highest for VNC.
- Update Update this tool to the latest version
- About raspi-config Information about this configuration tool
- Updating everything and install desktop XFCE via CLI and Putty:
- Read the guide here then follow the below listed super user commands using Putty at the CLI (command line interface):
- sudo apt update
- sudo apt upgrade
- sudo apt dist-upgrade
- sudo reboot (restart Putty)
- sudo apt install xserver-xorg
- sudo apt install xfce4 xfce4-terminal
- sudo apt install lightdm
- sudo reboot (restart Putty)
- Install VNC Viewer on desktop
- Connect to RPi via VNC Viewer - see screenshot
- Initial Python Setup:
- Read "How to Install Python 3" at RaspberryPi.org
- Read "Installing Python Packages" at RaspberryPi.org then follow the below listed super user commands using Putty at the CLI (command line interface). If any of these are already installed it is OK. On completion, reboot into the XCFE desktop via the VNC client and the Python 3 IDE (called IDLE) should now be in your Application Menu (under Developer Tools):
- sudo apt update
- sudo apt install python3
- sudo apt install idle3
- sudo apt install python3-pip
- sudo reboot
- Python Hello World and Blinky:
- Read RaspberryPiHQ - Blink RPi LED With Python
- Read Sparkfun Tutorial: Python and the RPi
- Read about the differences between using apt to install packages and using pip3 to install packages at raspberrypi.org. I don't really know what is best but I am going to try the following policy for now, if it is a unique or special Python package I will use pip3, if it is a general Linux package or a commonly used Python package, I will use apt. We may make exceptions to this policy for necessity or convenience.
- Read Headless RPi and Blinky Using Python at FreeCodeCamp and GPIO Zero Python Library then follow the below listed steps to get the Python "Hello World and Blinky" program running:
- Login to the CLI with Putty and enter the following to verify Python 3 is being used by the OS: python --version
- If it is not version 3 (mine was not) enter the following to open the "bashrc" user configuration file with Nano: nano ~/.bashrc
- Then scroll/arrow to the bottom of the "bashrc" file (don't change anything else) and add the line: alias python='/usr/bin/python3'
- Use Ctrl-X to exit and Y to save. Logout of Putty, reboot the RPi (I use the USB power switch) and login via Putty and check the Python verion the OS is using. Mine was now 3.7.3.
- You can also verify the pip3 version you installed earlier is OK using: pip3 --version. PIP is the built-in package manager for Python libraries. It is, IMHO, very complicated (Pip Home). But for our purposes, we will simplfy things by only using Python 3, so use pip3 whenever pip is used in an example.
- Install the T-Cobbler (RPi OFF) and plug into your lomgest breadboard. Turn RPi on Verify the 3.3 v, 5 v, and Gnd pins with your multimeter. (see pic of Mr. C.s T-Cobbler while reading a 3.3 vdc pin)
- As shown in the FreeCodeCamp tutorial above, connect the longer lead (+) of an LED to GPIO Pin 25 (conveniently labeled 25 on T-Cobbler) and the shorter side of the LED (-) to the GND rail on your breadboard. Connect a GND pin on your T-Cobbler to the the same GND rail on your breadboard.
- To access the GPIO pins, of the several gpio packages available the FreeCodeCamp tutorial uses gpiozero, and uses apt to install, so install that via Putty and PIP3 using: sudo apt-get install python3-gpiozero. Note: the corresponding pip3 install would use: sudo pip3 install gpiozero.
- Finally we are ready to leave the CLI and use the VNC Viewer and do the rest of the steps to create a pi user folder for Python programs that will automatically have the right file permissions to write it and torun it. Then we will write and run the blinky program using IDLE. The essential steps are to launch VNC Viewer, log into the pi and get to the XFCE desktop. Use the XFCE File Manager to create a subfolder under the pi/home folder called bin. Open IDLE, copy/paste our blinky program test file in a "New File" (see Python text file code here), then Run it! It worked first time. See my video below!
- Watch Mr. C.'s Video: Writing/Running a Blink LED program via VNC Viewer (on Windows 10) connected to a Raqspberry Pi B Plus using the XCFE Desktop and IDLE.
Electronics Supersites and Suppliers:
Public Key Authentication:
IoT - Internet of Things Related:
Using Live Online Weather Data To Control RPi GPIO Pins
Sending Live Online Weather Data From Rpi to Google Sheets via Python
- Watch Mr. C.'s Video: Using Python-RPi -OWM API to Control LEDs - python script owm_blinky program
- Explore: OpenWeatherMap.org (OWM)
- Read: Weather Data and RPi via Open Weather Map (OWM) API
- Read: How To Use an API (Application Program Interface)
- Read: How to use the OWM API with Python - Medium.com
- Explore: PyOWM Library Home - Python Wrapper for OWM and PyOWM docs and PyOWM Projects.
- Explore: PyOWN Code Recipes
- Explore: Weather Alert OWM API - aka triggers (not used yet)
- Explore: raw data files used at OWM (the city.list file is where I got the Oneonta and Ashville data). Best Windows program for unzipping "gz" files is 7-Zip (great program for any compress/expand tasks)
- Register at OpenWeatherMap.org and obtain a "Current Weather Data" API key (the API Key will be sent to your email upon registration)
- Login to RPi via Putty and install the Python 3 Library for OpenWeatherMap using "sudo pip3 install pyowm"
- Wire 3 LEDs to GPIO pins 23, 24, and 25. I used Green, Yellow, and Red 5mm LEDs respectively. Each is wired the same as in the Blinky sketch using 220 ohm current limiting resistors for each LED.
- Login to RPi via VNC and write the "owm_blinky" Python3 program using IDLE. Save in pi/home/bin folder. Use your API and change location stuff to Ashville of course. Run it from IDLE.
Using Rpi/Python to Read Cells in Google Sheets
- Watch Mr. C. 's Video: The headless Raspberry Pi running a Python program and posting real time weather data (the outside temperature for Oneonta, Alabama from Open Weather Map (OWM) in degrees Fahrenheit) to Google Sheets and displaying the data in a dynamic graph. And it is all free. Data storage comes of age, imagine the possibilities! (ref. Google Sheets API Python Quickstart):
- Explore Google API docs:
- Explore Related Projects:
- Explore Best Related Project: Chemistry LibreTexts 1.6: Writing to Google Sheets
(Setting up a Google Sheet to Accept Raspberry Pi Data)
Following the lesson above:
- Using your Chrome browser, Go to your Google Drive and open a new "Sheet". Label the first column "Time" and the second column "Temperature", give the Sheet a a title, I made mine: "RPi_OWM_GS". Note: OWM = Open Weather Maps, GS = Google Sheets. Copy the URL for the Sheet, mine is: "https://docs.google.com/spreadsheets/d/XXXY_your_long_sheet_url_goes_hereXXX/edit#gid=0". Keep "Sharing" set to Private (can change this later).
- Open a new tab in Chrome and go to the The Google API Console. Select "New Project" and name it: I called mine "RPi OWM Data". Copy the automatically assigned Project ID. Mine was: "rpi-owm-data".
- After reloading the Google page and selecting the new project. Click "Enable APIs and Services" and enable the Google Sheets API (look again at the above lesson if needed), Put "sheets" in the Search box that appears to find the link for the Google Sheets API.
- Use the "Create Credentials" button to create them for the Google Sheets API. Select "Google Sheets API" and "Other non-UI(e.g.cron job, deamon)" for type. For the "data to be accessed" select "Application Data" and "No" for the "app or compute engine question". Click the "What credentials do I need?" button and add the following credentials:
Service Account Name: MyRasPi,
Role: project \ Owner,
Key Type: JSON.
Make a copy of your Sevice Account Name and ID, mine were: Service Account Name: MyRasPi Service Account ID: myraspi @rpi-owm-data.iam.gserviceaccount.com
- Click "continue". You will get an automatically downloaded JSON key file for later. It will be in your downloaded files folder. Do not lose this. Open this file with Notepad or another program editor (I used Sublime) and copy the client email link. Go back to the original Chrome tab with the 2-column Sheets file created earlier and click "Share" in the upper right hand corner. Paste the email address and click "Send" to connect it to the Sheets file. Note: this action caused an email to be sent to my gmail account and it threw an "undeliverable message" error. It doesn't seem to mean anything or have an adverse effect I can find at this point.
- Go back to the Google API's tab in Chrome and Click on ENABLE APIs and SERVICES again, we still need to enable the "Google Drive API" so put that in the search box and enable the "Google Drive API". No new credentials should be needed so this is easier than the Google Sheets API.
- Copy that JSON file you downloaded from Windows into the directory where you write your python programs on your RPi. You can rename the file, the extension must be ".json". I renamed mine "rpiowmdata.json". I don't know if "WinSCP" (Windows Secure Copy per wiki) is the best tool for transferring files to our headless RPi but I have used it in the past and found it very nice. It is made for transferring files, whereas Putty is made for executing command line instructions. They both are very good at what they do. I much prefer using a GUI like WinSCP to transfer files instead of using the command line. It can be a chore to get Windows and Linux file systems to cooperate. Get WinSCP for Windows here. Here is a WinSCP demo video: "How to copy files to Raspberry Pi over Network from Windows". Of course SSH is already enabled and we know the RPi IP so you can skip those steps. All you need to connect the Windows PC to the RPi is the Rpi's IP address and your RPi user name and password. I was able to use WinSCP to upload the "rpiowmdata.json" from Windows to the RPi directory we are using for our Python programs: "/home/pi/bin".
- Make sure the RPi is current, run the following at the RPi command line via Putty:
- sudo apt-get update
- sudo apt-get upgrade
- Install the Python libraries (goto links for details) needed to connect to Google Sheets
(oauth2client is deprecated but works, check out oauthlib for future use)
(gspread authorization documentation fyi):
- sudo pip3 install gspread
- sudo pip3 install oauth2client
- sudo pip3 install google-auth-oauthlib
(Note: I changed this line from the lesson above that I took it from, I had errors pointing to this line until I removed "--upgrade", perhaps it assumed it was installed by default already.)
- Use VNC Viewer on Windows to login to the headless RPi. I then used the RPi desktop File Manager to copy the previous RPi project Python program that comnected the RPi to Open Weather Map (mine was named: "owm-blinky")and renamed it "sheets_owm_blinky.py" program. Then I modified it to append Oneonta's temperature data obtained live from Open Weather Maps to my Google Sheets file. Here is a text file of the Python program that worked after a couple of tweaks: sheets_owm_blinky,py
- Explore Annotated Screen-shot:
Google Sheets in Process of Storing Rpi OWM Temperature Data
- You can add a dynamic chart (one that automatically adds new row data to the chart as data is received from the RPi) by editing the cell range driving the chart. For my chart, the data range when I first inserted the chart (click on an empty cell and select "Insert Chart") was from "A1:B17". All you need to do to make the chart dynamic is to set the range to "A1:B" (i.e. delete the number after "B"). Thanks for this post on StackExchange for that tip.
Connecting an Arduino Uno to a Raspberry Pi 3+
- Add statistical summary data to the Google Sheet used to store Open Weather Map temperature data. Add a
cell for the following descriptive statistics using a cell reference of, for example, "B1:B", to make the data in column B be dynamic, i.e. to automatically update when new data is appended to the sheet by the Raspberry Pi Python program. I added seven cells: standard deviation, mean, median, Q1 and Q3 quartile medians, minimum and maximum.
- Use VNC Viewer, log into the RPi and use the file manager at the RPi desktop to copy the previous Python program used to append Open Weather Map (OWM) temperature data to Google Sheets and rename it. It has most of the functionality we need so we can just add the feature to read the 7 cells with the statistical summary data. Make sure it is in your /pi/home/bin/ folder. I named mine "sheets_owm_stat_summary.py". Now edit the program to read the cell data. Reference the gspread library as needed. In my Google Sheets file, the 7 cells that contain the 7 "live" statistical summary summary results are in cells D24 to J24.
- A really cool discussion of the possible pitfalls of relying on summary statistics to tell the whole story, and how a statistical measure, to be meaningful, has to be accompanied by (to use Dr. Deming's words) a "profound knowledge" of the process that generated the statistics, check out the "Datasaurus Dozen" aka "How the Same Statistics Can Produce Widely Different Graphs" (click here).
Arduino To/From RPi and RPi To/From Google Sheets
- Caution: At no time connect the 5vdc Arduino GPIO pins directly to the 3.3vdc RPi pins.
- How to Connect RPi to Serial USB Port with Python via CLI - at OpenSource.com
- Serial Communication between RPi & Arduino - at WUSTL.edu
- Obtain a USB A Male (the Arduino USB port) to USB B Male (the RPi port) cable. These are very common and is the same cable you already use to connect the RPi to your desktop PC and most of the time you will want to disconnect from the PC and plug into the RPi. But if you want both connected, or just find using the desktop cable inconvenient, and you don't have an extra cable already, here is one for $3 on Amazon:
- USB A Male to USB B Mable 1.5 ft. Cable - Amazon
- Determining the serial port that connects the RPi and Arduino by using Putty at the RPi CLI (Command Line Interface aka "the terminal"):
- Note: connecting the USB ports is OK as USB ports are always 5vdc on any device.
- With the RPI and Arduino NOT connected -- at the RPi CLI use: "ls /dev/tty*" to list all "tty" ports. Turn OFF both the Arduino and the RPi.
- CONNECT the Arduino and RPi via the USB A (Arduino) to USB B (RPi) cable. You can leave the Arduino connected to your PC so you can observe its serial monitor output or unplug the USB B cable and plug it into the RPi. You will be able to monitor what serial input the RPi is receiving from the Arduino using the Python program output when you run the finished program from VNC VIewer on your desktop.
- Power up the Arduino via the 9 vdc supply then power up the Rpi. Using Putty, at the RPi CLI use: "ls /dev/tty*" to determine what new "tty" port appears, this is the "tty" port that is being used for Arduino-RPi serial communication.
- The port on my RPi was "/dev/ttyACM0". To see my connected and not connected "ls /dev/tty" results click here
- Using Putty at the RPi terminal, install Pyserial, the Python Serial library using:
- python -m pip install pyserial
- Sending "Hello World" from the Arduino to the RPi via the USB Serial connection:
- Plug in the 9vdc supply for the Arduino Uno and connect the Arduino via USB to your desktop PC and install the following "Hello World" program for the Arduino using the Arduino IDE. (text file with Arduino Uno and RPi programs). You can disconnect the RPi from the PC when done after checking that the "Hello World" is being sent using the serial monitor feature of the Arduino IDE.
- Go the the headless RPi desktop via the VNC Viewer and install the following "Hello World" Python program for the RPi using IDLE, the Python editor built into Python. (text file with Arduino Uno and RPi programs).
- Connect the Arduino Uno and RPi B+ using the USB A (Arduino) to USB B (RPi) cable.
- With the "Hello World" program running on the Arduino, run the new "Hello World" Python program on the RPi. You should see the "Hello World" message sent from the Arduino.
Use 0.96 in. 128x64 OLED Display with RPi
- Watch Mr. C. 's Video: Arduino To/From RPi and RPi To/From Google Sheets
- This project builds on prior Arduino and "headless" Raspberry Pi projects and demonstrates using the Arduino to get local inside RH and temperature data via a DHT11 sensor and sending this data to the RPi via the serial port (USB A port on Arduino to USB B port on RPi) and using the RPi to get local outside temperature via ethernet to the internet and Open Weather Data (OWD) and sending this data both to the Arduino via the serial connection and to Google Sheets via the internet. The Arduino will use a small (0.96 in) OLED display to show the data and Google Sheets in the Windows desktop will also show the data. The RPi would be a better choice to drive the OLED display but I wanted an excuse to send actual data, not just text, from the RPi to the Arduino.
- 2-Way Comm. - RPi and Arduino - at AutomaticAddison.com
- Raspberry Pi Arduino Serial Communication - Roboticsbackend.com
- OLED I2C Display Arduino/NodeMCU Tutorial - instructables.com/
- I2C Tutorial - at Sparkfun
- I2C Tutorial - at CircuitBasics.com
- I2C Scanner - sketch
(run this Arduino sketch to get I2C address for connected devices)
- Key Arduino functions:
- Declaring String Data Types
- Strings, Unicode, and Bytes in Python 3
- Unicode-UTF8-and Character Sets-The Ultimate Guide - Smashingmagazine.com
- Why are hexadecimal numbers prefixed with 0x?
- ASCII and UTF-8 Tables
- UTF-8 - at W3Schools.com
- Key Python Libraries/Methods:
- Pyserial Library - the Python Serial library (should already be installed)
- Python String rstrip() Method
- Python String encode() Method
- Python String decode() Method
- Build Arduino circuit and sketch - text version of Arduino sketch
- Build RPi circuit and sketch - text version of Python program
- *Using I2C OLED Display With RPi - raspberrypi-spy.co.uk (* I used this a lot)
- Adafruit I2C via CircuitPython:
- Adafruit Python Library for OLED Displays
- CircuitPython and RPi
- Installing CircuitPython Libraries on Raspberry Pi
- pip3 install Adafruit-Blinka
- Example - Using Circuit Python with BME280 Temp-RH Sensor
- Python Installation of SSD1305 Library
(inc. steps to change the I2C to 1MHz if needed. Default is 100KHz or 400KHz.)
- *Python OLED Example
- I2C Part 2 - Enabling I²C on the Raspberry Pi - abelectronics.co.uk
- Raspberry Pi Tutorial Series: I2C - waveshare.com
- Python Imaging Library (PIL)
- I2C Drive - hardware FYI
- From Putty CLI on RPi Update Linux OS:
- sudo apt update (FYI: mine updated 4 packages)
- sudo apt upgrade (FYI: mine upgraded 33 packages)
- sudo apt autoremove (FYI: mine removed 1 packages)
autoremove is used to remove packages that were automatically installed to satisfy dependencies for other packages and are now no longer needed. Packages manually installed are not autoremoved. If you ask apt to install a package you already have installed, it won't do anything but will now set it as manually installed.
- From Putty CLI on RPi Verify I2C Enabled:
- sudo raspi-config (select "interfacing options" to enable I2C)
- From Putty CLI Install and or Verify Python Libraries:
(Note: I was able to cut/paste each command from Windows to the CLI)
(use Ctrl-C to copy and right click to paste, -y assumes "yes" to all prompts)
- sudo apt install -y python3-dev (mine was already installed)
- sudo apt install -y python-smbus i2c-tools (new install)
- sudo apt install -y python3-pil (new install Python Imaging Library aka pillow)
- sudo apt install -y python3-pip (mine was already installed)
- sudo apt install -y python3-setuptools (mine was already installed)
- sudo apt install -y python3-rpi.gpio (mine was already installed)
- Power down the RPi and hook up the OLED. Use 3.3 v, Ground, GPIO 2 (SDA) and GPIO 3 (SCL)
See picture - Mr. C.'s OLED to RPi Connection
- From Putty CLI run "i2cdetect -y 1" where "1" should be the name used by default for the RPi B+ I2C bus. This will confirm the OLED display is powered up and is transmitting its I2C address. Mine returned 3c which represents hex 0x3c where the 0x is the tag used by C to indicate hex. see my RPi CLI screenshot
- From Putty CLI run "pip3 install adafruit-blinka" to install the Adafruit Circuit Python libraries for I2C and SPI (we aren't using SPI currently). This will install a few different libraries such as adafruit-pureio (our ioctl-only i2c library), spidev (for SPI interfacing), Adafruit-GPIO (for detecting your board) and of course adafruit-blinka. CircuitPython libraries and adafruit-blinka will work on any Raspberry Pi including the original 1, the Pi 2, Pi 3, Pi 4 or Pi Zero. Adafruit_blinka (named after Blinka, the CircuitPython mascot) translates the CircuitPython hardware API on the Raspberry Pi to the python RPi.GPIO library. Any I2C interfacing will use ioctl messages to the /dev/i2c device. For SPI will use the spidev python library, etc. These details don't matter so much because they all happen underneath the adafruit_blinka layer. Any code for CircuitPython should run on the Raspberry Pi.
- From Putty CLI run "pip3 install adafruit-circuitpython-ssd1305" to install the Circuit Python library for the SSD1305 module. The AdafruitSSD1306 Library Documentation Release 1.0 May 10 2019 states: "Adafruit CircuitPython driver for SSD1306 or SSD1305 OLED displays. Note that SSD1305 displays are back compatible so they can be used in-place of SSD1306 with the same code and commands."
- Make and install your "Hello World" OLED Demo Python program. I used the Sublime text editor to tweak the example program provided on Adafruit's SSD1305 module page. Make sure you use the I2C address the I2CDetect program returned for your OLED display. Edit the display size (pixel width/height) if needed, mine is 128x64. Upload your Python file via the WinSCP app from your Windows desktop to the "home/pi/bin" folder on your RPi. Here is a text copy of my Python program: demo_i2c_128x64_oled.txt
- Login to your headless RPi via VNC Viewer and run the Python your "Hello World" OLED Demo. Voila. see picture