@[email protected]

RaspberryPi

HomeLab

Motivation

I became more interested in personal privacy after my Roku started spying on what we were watching outside the Roku itself, our mesh WiFi router switched to a subscription model for “AI” and “cloud” features, and our smart home switches required access to “the cloud” just to turn on lights. TVs, WiFi routers, and smart home devices are all driving prices down by supplementing hardware sales revenue with personal data sales.

On top of that, after creating a custom smart lock, I saw first hand how Google and Amazon's smart home infrastructures are built around selling cloud services and capturing my personal data, while Apple HomeKit is designed to work without any internet access at all.

Given these consideration, I wanted a more robust router and firewall between my home network and the internet. I wanted to be able to completely block smart home devices from accessing the internet. And I wanted to do everything as cheaply as possible while maintaining compute resource (CPU, RAM, disk) separation between self-hosted services.

Setup

The Ubiquiti EdgerRouter X is the router and firewall while the mesh WiFi is in “bridge” mode, effectively operating as a switch. IP addresses are assigned in ranges and firewall settings are used to block all devices from the internet except those that need it (Apple TV, Laptops, Phones, etc.).

Four Raspberry Pis host all self-hosted services. Public services like code hosting, federated social networks, and a bitcoin node. And private services like DNS-based ad and tracker blocking. (After using Pi-Hole for a while, I switched to AdGuard Home, which is just simpler and easier to maintain.)

Finally, power over Ethernet (PoE) with a PoE switch is used to reduce the cords to the Raspberry Pis.

Custom Racking

A downside of not using a standard rack-mounted host is the non-standard form factors of the Raspberry Pis and hard drives etc.

To handle this, I 3D printed a Raspberry Pi 2U rack mount. It's not used in a rack configuration but it's actually just a great way to have easy, uniform, and modular access to the Pis.

Hardware was purchased from McMaster-Carr.

For the hard drives, I designed and 3D printed a custom stand.

Configuration Management

Host configuration is managed with Ansible. The roles are written to be minimally invasive and optimized for low maintenance.

The Ansible roles are open source.

Layout

For clarity, my specific Raspberry Pi Ansible playbook in provided below:

- hosts: rpis
  roles:
    - rpi-base
    - apt-cacher/client
    - prometheus/rpi-client

- hosts: admin.local
  roles:
    - adguard-home
    - apt-cacher/server
    - prometheus/server

- hosts: btc.local
  roles:
    - block-device
    - bitcoind
    - lnd
    - bitcoind-prometheus-exporter

- hosts: media.local
  roles:
    - block-device
    - plex
    - transmission
    - homebridge
    - minecraft
    - nginx

- hosts: web.local
  roles:
    - block-device
    - postgresql
    - pleroma/aws-s3-backup
    - pleroma/otp
    - writefreely
    - mercurial/aws-s3-backup
    - mercurial/web
    - oragono
    - prosody
    - nginx

Conclusion

Using a hardwired router as the articulation point between the internet and the rest of a home network is a great way to get privacy, security, and self-hosting without really investing much.

#RaspberryPi #SelfHosting #Homelab #Linux #RaspberryPi #Homekit

There seem to be two schools of thought in the homelab community regarding whether Raspberry Pi clusters are worth it. Many are of the thought that four Raspberry Pis, clustered, with all the peripherals like hard drives, networking equipment, power supplies, etc. adds up to be more expensive than a single NUC or blade with way better specs. This seems reasonable and yet I still find myself drawn to the Pi cluster.

Why? Because of resource isolation and balance.

I have four Raspberry Pi 4 B (4gb) hosts running Ubuntu 64bit. I can have one dedicated to running a Bitcoin full node and know that any disk or CPU spike will not interrupt my web hosting on a separate host. I have dedicated and isolated units comprising CPU, RAM, NIC, and HDD resources. This, in some ways, is the opposite of running a true cluster with tools like Kubernetes. In a true cluster environment, the system is agnostic to where services are deployed on the bare metal, with some caveats.

Under normal loads, my Raspberry Pis are not maxing out any system resources except for network bandwidth at times. A perfect case for a CPU per NIC. It's also why I won't be adding an 8gb Pi 4 to my homelab any time soon.

Some other reasons people go with a Pi cluster, unverified by me, are lower power usage, substantially lower costs in some cases, and above all else, it's just fun.

#RaspberryPi #SelfHosting #Homelab #Linux

GPU Transcoding with Raspberry Pi

Wyze Cam on Apple HomeKit using the Raspberry Pi 4 hardware accelerated transcoding.

Wyze Cams are inexpensive and awesome web cams. Unfortunately Wyze will not support Apple HomeKit on the current cameras. An alternative is to use Homebridge on a Raspberry Pi to “bridge” the cameras into HomeKit.

NOTE  —  16th of Feb, 2020: A YouTube video by Tech Craft explains how to check an RTSP stream for native HomeKit H.264 support. This is luckily the case for Wyze Cams and therefore there is no need to transcode the stream with h264_omx as this post originally described. The post has been updated to use the more performant and simpler stream copy stream technique.

Performance Notes

  • The Wyze Cam RTSP output format is H.264 encoded which is the stream coded required by HomeKit so no transcoding is required.
  • The Raspberry Pi 4 has a lot of hardware updates to support higher bandwidth video processing. This may also work on older Raspberry Pis. The RPi 4 supports H.265 (4kp60 decode) and H.264 (1080p60 decode, 1080p30 encode).
  • While live streaming with the setup described here, memory usage peaked at 200Mb and CPU peaked at 50% on a single core.

Prerequisites

  1. Install the Wyze Cam RTSP firmware
  2. Get a Raspberry Pi 4 B 1GB with Raspbian Buster or newer
  3. Install Homebridge

Setup

This is where experimentation was needed to find a successful setup.

On the Raspbian Buster apt repos all dependencies are ready to go with no custom compiling required.

Install the following packages with the respective package manager:

  • apt: libavahi-compat-libdnssd-dev
  • apt: ffmpeg
  • npm: homebridge-camera-ffmpeg  —  The fork homebridge-camera-ffmpeg-omx is not needed because the stream does not need to be transcoded.

Finally, configure Homebridge for each Wyze Cam. The key here is to use the copy vcodec to copy the native H.264 video stream strait to the HomeKit stream.

This config also has the combination found to work best with both streaming and snapshotting for HomeKit. Check out the homebridge-camera-ffmpeg docs and defaults before adding unnecessary configuration options.

Example Homebridge config.json:

{
    "bridge": {
        "name": "Homebridge",
        "username": "12:34:56:78:90:AB",
        "port": 51900,
        "pin": "031-45-154"
    },
    "description": "Homebridge",
    "platforms": [
        {
            "platform": "Camera-ffmpeg",
            "cameras": [
                {
                    "name": "Wyze Cam",
                    "videoConfig": {
                        "source": "-i rtsp://username:[email protected]/live",
                        "stillImageSource": "-i rtsp://username:[email protected]/live -vframes 1 -r 1",
                        "vcodec": "copy"
                    }
                }
            ]
        }
    ]
}

Confirmation

At this point, it should be possible to add the accessories to the Home app and see both smooth live streaming and preview snapshots from the cameras.

On the Raspberry Pi, run top or htop to confirm the load is not on the CPU while streaming.

#WyzeCam #HomeKit #RaspberryPi #Homebridge

Apple HomeKit with Raspberry Pi

I used a Raspberry Pi and Pimoroni Automation pHAT to make my 1970s (?) apartment intercom system Siri controlled and internet accessible. The full project code is available on a src repo.

Final build mounted and ready to be reinstalled

The Problem

My apartment building lobby door lock is controlled by an intercom panel in my unit. To unlock the door, the switch completes a circuit to an electromagnet in the lobby. And for someone to ring my unit's doorbell, a switch in the lobby completes a circuit to power a really annoying buzzer in my unit. I can also talk and listen to a speaker in the lobby.

If I'm at work I can't get a package delivered or receive guests. Also, I'd been looking for a fun Raspberry Pi project for a few years and finally it struck me: use the Raspberry Pi to control these switches.

Intercom Layout

To start, I took the panel off the wall to see how it worked and used a voltmeter to test what all the wires mapped to and at what voltages. I only cared about two functions: unlocking the lobby door and receiving a doorbell signal. There are five wires and four functions, leaving the fifth wire as ground, all at 52V:

  1. Unlock door
  2. Listen for doorbell
  3. Talk to lobby speaker
  4. Listen to lobby speaker
  5. Ground

Raspberry Pi

When selecting a Raspberry Pi or Arduino to control these circuits, I went with the Raspberry Pi because it's a full linux computer and seemed the most flexible. I picked the Raspberry Pi Zero W for the smaller form factor and builtin WIFI. SSH access is provided via ethernet emulation over one of it's USB ports.

I bought the following for one day shipping from Amazon:

Raspberry Pi Add-On Board

Now I needed a way to switch and read the intercom circuits. “HAT” is the name of a full-sized add-on board and “pHAT” is the name of the Pi Zero form factor add-on board. Pinout is a great resource with a fairly comprehensive list of pHAT boards. I figured I needed a “relay” to unlock the door and a “analog to digital converter” to read the doorbell signal. One board fit the requirements perfectly:

Raspbian Setup

With the hypothesis of how all this would fit together, I bought the components and started on the software.

I followed two guides to set up a headless Raspian with WIFI enabled and SSH over USB emulated ethernet:

  1. Use Your Raspberry Pi As A Headless System Without A Monitor
  2. Connect To A Raspberry Pi Zero With A USB Cable And SSH

Writing the Software

With the hardware and OS sorted, I started experimenting with controlling the Automation pHAT using their online docs and Python function reference. I used LEDs and resistors to experiment with the functionality in isolation from the intercom.

pHAT Python Server

The pHAT Python API access must be single threaded but because I need to poll for the doorbell and for commands to unlock the door in parallel, I made a multithreaded Python server with a message queue for commands using stdio.

Doorbell

The doorbell goes into the Analog-Digital-Converter (ADC) and a resistive divider is used to bring the 52V into the requisite 24V range. The ADC digital reading of 6.0—6.3 is when the doorbell is ringing for whatever reason. Everything else is noise. Here's some example output from a one second polling Python test:

7.65
7.65
7.62
7.62
6.08  <-- doorbell starts ringing
6.18
6.19
6.18
6.19
7.62  <-- doorbell stops ringing
7.62
7.62
7.63
7.63

Door Lock

The door lock is controlled by the single relay available on the pHAT which doesn't require bringing the voltage down with a resistor.

Apple HomeKit Integration

I then created a HAP-NodeJS accessory to expose the Door Python server to Apple HomeKit Protocol (HAP) for Siri and Home App control. I use systemd to keep the HAP server running. I also use the HAP accessory to timeout the door being unlocked and relock it. This is to mimic the behavior of the physical switches which spring back to neutral (locked) when not being held.

Representing a door lock with HAP-NodeJS in HomeKit was easy but getting the doorbell event to work was very fragile. Now I get iOS notifications when the doorbell rings!

Doorbell Ringing iOS Notification

It was also extremely difficult getting NodeJS and all the dependencies installed correctly. Once I got everything installed and working, I wrote Ansible roles to codify exactly what is required.

See the src repo for the full install layout.

Final Build

I was able to attach the final buildout to the preexisting circuit board screw terminals so the intercom panel still works as before but with the addition of Siri integration! Siri and Apple HomeKit turned out to be a huge win. It meant I didn't need to run any external servers to expose my services to the internet, write any mobile apps, or do anything special to integrate with Siri. Alexa was much more complex and I ended up removing that support.

I was able to use my Apple TV to bridge all of my HomeKit devices, including my custom Lobby Door, to my iOS devices through the internet. No more missed packages and a fun project over the holidays.

Future Tasks

Right now the Automation pHAT must poll for doorbell events. I believe there is a way to use the Raspberry Pi GPIO pins to create an event-driven approach.

Perhaps I'd also like to attempt to get the “talk” and “listen” functionality working with an additional audio-focused pHAT.

HomeKit supports BTLE and I believe the Raspberry Pi Zero W does as well. It would be nice to use the lighter-weight protocol instead of WiFi.

#RaspberryPi #Homekit #HomeAutomation #Python