Trying out vector embeddings

newsSum is a Google App Engine application that bundle articles from different news sources. To try out ML embeddings, I decided to add a suggestions service.

High-level idea

• There will be no changes to the backend of "newssum". The suggestion service "newssum-sug" will be implemented as a separate service
• The frontend of "newssum" will check if the suggestion service "newssum-sug" is available. If so, it will allow the user to expand an article and query the suggestion service for additional information to display

Implementation of the suggestion service

• Technically, "newssum-sug" could gather suggestions from any sources (e.g. Google search results, a Youtube video etc). But for now, it will process articles from selected "newssum" sources. So, there will be scheduled tasks to collect articles from "newssum" and prepare them for searching.
• Vector embeddings will be used to find similar articles. A machine learning model is used to turn a news headline into a vector of numbers. When a query comes in, an embedding will also be generated from that query. By comparing the distance between vectors, we could find articles that are related to the query.
• The embeddings generated during batch processing are stored in a vector database. The database will also provide the mechanism for searching vectors by distance.
• Since "newssum" is for current news only, embeddings will only be kept for 2 days.
• The suggestion service can also be used for free-text search. But for now, the frontend only uses it for article suggestions.

While "newssum" is open source, the "newssum-sug" service is still under development in closed source. But the basic functionality has been integrated and available on the demo site.

A machine learning model to identify music albums from photos

 

I was looking for a home automation project to select and play specific music album from stream services. There are similar ideas of using NFC tags. Basically, it means preparing some NFC tags with album/movie cover arts on them. And putting a tag on the reader will trigger the playback of that album/movie. While it brings the joy of handling and selecting physical collections, it costs money and time to prepare those NFC tags and I wanted to avoid that.

Since now we have those machine learning models and classifiers, I thought I can just train up a model to look at a webcam photo of a record / CD and tell me the Spotify link to play that album.

BTW, I know Microsoft co-pilot (or maybe OpenAI too) can do it without any special training, but then I don't want to pay extra for that and just wanted to host the model on my own machines.

I imagine it will be something like this:

I put an album in front of a webcam...

... and the model will tell me the Spotify URL to pass on to the music streamer

Long story short, my model can a identify my music collection with a 98% correctness (more on that later). If you are interested in the technical details and the scripts used to train the model, they are available on github: https://github.com/kitsook/AlbumSpotter

But eventually I didn't integrate this into my home automation, which is kind of related to the correctness. When I got a new CD / vinyl record, I always add that to my collection on Spotify. So I can just get the cover arts from Spotify to train my model. But then I discovered there are at least two problems that will affect the correctness:

• there are many editions of the same album. e.g. I could have a physical CD of the standard edition but my Spotify collection has the extended edition with a different song list
• nowadays artists tend to release an album with several "special" cover arts. My physical copy could look totally different from the one on Spotify

That means I will need to cleanup the data for a more accurate result. As procrastination kicks in, I am stopping the project with just the machine learning model and the home automation part will be a future project.

OpenSUSE Tumbleweed - another bug

 

Tumbleweed used to be great in the past eight years or so that I used it on my main desktop. But recently it is just bugs after bugs after every update.

Here is another one after updating today: the KDE screenlocker crashed with segfault and failed to unlock the screen. Seems to be pam library related.

Google cloud AppEngine deployment issue with cache

Notes to self: when deploying Google Cloud applications, if encounter the following "no such object" error, it is because the deployment process is looking for cache that doesnt exist.

"build": ERROR: failed to initialize analyzer: getting previous image: getting config file for image

To solve it, deploy with "--no-cache" parameter.

Segfault with bluez and A2DP

Currently (as of Feb 23, 2024) openSUSE Tubleweed with bluez 5.71-2.4 is experiencing segfault whenever I try to connect my Sony WH-H800 headphone (but fine with others).

After some digging, seems like it is plagued by a recent bug. Waiting for Tumbleweed to include the patch.

[Fri Feb 23 16:23:24 2024] bluetoothd[9971]: segfault at 5600bf0628b5 ip 00005605de5743c1 sp 00007ffee8f18f70 error 4 in blu
etoothd[5605de552000+d6000] likely on CPU 7 (core 1, socket 0)
[Fri Feb 23 16:23:24 2024] Code: 41 83 2c 24 01 0f 85 1b ff ff ff 4c 89 e7 e8 96 f4 fd ff e9 0e ff ff ff 90 41 55 41 54 55 5
3 48 83 ec 08 48 8b 2a 48 8b 7a 08 <48> 8b 45 20 4c 8b ad 88 00 00 00 4c 8b 20 48 85 ff 74 19 c7 47 08

$ sudo coredumpctl info 9971
          PID: 9971 (bluetoothd)
          UID: 0 (root)
          GID: 0 (root)
       Signal: 11 (SEGV)
    Timestamp: Fri 2024-02-23 16:23:24 PST (4h 2min ago)
 Command Line: /usr/libexec/bluetooth/bluetoothd
   Executable: /usr/libexec/bluetooth/bluetoothd
Control Group: /system.slice/bluetooth.service
         Unit: bluetooth.service
        Slice: system.slice
      Boot ID: 153713284dde4d7cba57f31e2956690d
   Machine ID: 5c42528e25094a3cb1af7e2c43a85357
     Hostname: linux-lct7
      Storage: /var/lib/systemd/coredump/core.bluetoothd.0.153713284dde4d7cba57f31e2956690d.9971.1708734204000000.zst (present)
 Size on Disk: 138.1K
      Message: Process 9971 (bluetoothd) of user 0 dumped core.
                
               Stack trace of thread 9971:
               #0  0x00005605de5743c1 n/a (bluetoothd + 0x463c1)
               #1  0x00005605de55f4d0 n/a (bluetoothd + 0x314d0)
               #2  0x00005605de55f5a8 n/a (bluetoothd + 0x315a8)
               #3  0x00005605de569767 n/a (bluetoothd + 0x3b767)
               #4  0x00007fc0b22daf30 n/a (libglib-2.0.so.0 + 0x5bf30)
               #5  0x00007fc0b22dcb58 n/a (libglib-2.0.so.0 + 0x5db58)
               #6  0x00007fc0b22dd42f g_main_loop_run (libglib-2.0.so.0 + 0x5e42f)
               #7  0x00005605de5555dc n/a (bluetoothd + 0x275dc)
               #8  0x00007fc0b1e2a1f0 __libc_start_call_main (libc.so.6 + 0x2a1f0)
               #9  0x00007fc0b1e2a2b9 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2a2b9)
               #10 0x00005605de5566b5 n/a (bluetoothd + 0x286b5)
               ELF object binary architecture: AMD x86-64

Logitech Media Server commands

While moving the music stream service from Tidal to Spotify on Home Assistant, I found I forgot how to get the playlist ID from Logitech Media Server (LMS).

Note to self: connect to LMS with "nc lms-address 9090" and issue the command "playlists 0 100 tags:E". The command doc is available at "lms-address:port/html/docs/cli-api.html"

Solving the Knapsack Problem with recursion in Java

Source available on Github.

Wikipedia has detailed description on the Knapsack problem and pseudo code on solving it. However, while implementing it with Elixir, seems that using simple recursion yields cleaner code. This is not the fastest way to solve the problem, but just easy to understand.

Basically, three return points within the recursion:

• if no items, result is 0 and return
• pop the first item. if item weight is over the max limit, return recursion result of the remaining items
• return the max between (current item value + recursion result with the remaining item) or (just the recursion result with the remaining items only)

However, not only it is inefficient, recursion will cause stack overflow error in Java. See the last (disabled) test case. Use -Xss parameter to increase stack size and run it for fun.

Caffeine (Simplified)

 

Found my old Turbo Pascal book on the bookshelf. Just so happen I was searching for a Windows program to prevent screen lock and found the open source Caffeine also written in Pascal.

Forked the repository and fired up Lazarus. Revised the UI a bit. Now it starts as a tray icon and can be controlled by the right-click menu.

Source code and executable available on github.

The Sins of the Fathers

 

Heard someone mentioned about the author Lawrence Block and found the ebook The Sins of the Fathers available from the library.

Overall not bad. Kind of nostalgic reading about the 70s. Will definitely see if can find the whole series.

Here is a review from Bing Chat:

The Sins of the Fathers is a gripping and gritty introduction to Matthew Scudder, a former cop turned private eye who operates in the dark and dangerous streets of New York City. Scudder is hired by a wealthy father to investigate the murder of his estranged son and his girlfriend (Bing's mistake. This should have been "his estranged daughter and her roommate"), but soon discovers that the case is more complex and twisted than it seems. The book is a masterful blend of mystery, suspense, and character study, as Scudder delves into the lives and secrets of the victims and their families, while also confronting his own demons and flaws. The Sins of the Fathers is a classic of hard-boiled fiction that will keep you hooked until the end.

Self-hosted Wyze Cam v3 camera without internet access - Part 1

This is part 1 of the notes on how to self-host Wyze Cam v3, without internet access, as a RTSP camera. 

Here Part 1 will focus on preparing the network setup. Part 2 (TBC) will be on configuring the Wyze camera.

Background 

For testing purposes, I have a Home Assistant(HA) setup with several Zigbee devices running Zigbee2MQTT and Mosquitto.

To add NVR feature, I have since installed Frigate and added a Google TPU card for accelerating event detection. The IP cameras used are Wyze Cam v3 patched with custom firmware to output two RTSP streams (a lower resolution for preview and event detection, and a higher resolution for recording).

Here are some highlights of the original setup

• The ISP dual-band router supports 5GHz network
• The HA machine is setup in a separate room running openSUSE Tumbleweed. It is connected to an Access Point (AP) via LAN cable
• The AP is an old dual-bank router flashed with DD-WRT custom firmware. It serves two purposes: the 5GHz network is configured as a wireless bridge to provide internet access to the HA machine and allow other devices on the home network to access HA, while the 2.4GHz network (with a different SSID) is configured as access point for IP cameras

The setup runs fine with one issue: the IP cameras have internet access. I don't like the idea that the IP cameras can "phone home" anytime. Ideally, after initial setup, I would like to remove the Internet access from those cameras.

Tried to setup virtual networks, bridges, and firewall rules etc on the AP. But none worked reliably.

So here is the new idea:

• since I have several USB wifi adapters not in use, I could add one to the HA machine and configure it as an AP with a different SSID and subnet. It will be running in 2.4GHz to minimize interference with the 5GHz home network
• with the HA being a Linux machine, I will have the flexibility on routing (or not routing) traffic around i.e. effectively blocking internet access for the cameras
• I don't need to access the IP cameras remotely (i.e. when away from home), don't need to receive any notifications on my phone, and don't mind giving up using the Wyze app. If I really need to access the cameras remotely, I could VPN back to my home network first.

A few things needed to be configured on the HA Linux machine.

EDIT 2024-06-02 A few changes on the device setup:

• the rtl8812au package from OpenSUSE isn't stable and keeping up to date with kernel updates. So ended up installing the driver from source as dkms
• using NetworkManager to setup host access point seems to cause conflicts on many things. Ended up to unmanage the wifi adapter with nm (adding "unmanaged-devices=interface-name:wlp0s16f0u1;" in /etc/NetworkManager/conf.d/99-localhost.conf). Then force the ip on the device with ifconfig with a custom systemd service. Finally use hostapd for the access point setup

EDIT 2024-06-10 script for the custom systemd service, assuming the interface is wlan0:

[Unit]
Description=Assign IP address to wifi interface for AP
Requires=sys-subsystem-net-devices-wlan0.device

Before=hostapd.service dnsmasq.service
After=sys-subsystem-net-devices-wlan0.device

[Service]
Type=oneshot
ExecStart=/usr/bin/ifconfig wlan0 up 192.168.20.1 netmask 255.255.255.0

[Install]
WantedBy=multi-user.target

USB wifi adapter

The wifi adapter I will be using has the Realtek 8812 chipset. The driver can be installed via `rtl8812au` package. However, installing it caused the Coral TPU dkms driver stopped working. Eventually, needed to install the newer version and recompile.

Next is to configure the wifi adapter as AP. Since the openSUSE machine is using NetworkManager, I used `nmcli` to configure it. The steps are similar to those mentioned here for Centos.

EDIT 2023-11-18. Some addition useful commands:

- disable WPA: sudo nmcli con modify "connection name" 802-11-wireless-security.proto rsn

- disable WPS: sudo nmcli con modify "connection name" 802-11-wireless-security.wps-method 1

EDIT 2023-11-25. To disable the excessive logging of the driver, change the log level to none

For the examples below, assume the new wifi interface is "wlan0" and has the IP address of 192.168.20.1/255.255.255.0. The two IP cameras will be assigned with address 192.168.20.91 and 192.168.20.92.

DHCP setup

Each IP camera will be assigned the same IP address via DHCP service provided by dnsmasq. Following these steps for a minimal setup.

local-service

conf-file=/etc/dnsmasq.d/trust-anchors.conf

dnssec

# only serve dhcp on the wifi interface

interface=wlan0

# some dynamic ip in case needed to connect for troubleshooting

dhcp-range=192.168.20.80,192.168.20.89,255.255.255.0,6h

# fixed ip for the ip cam

dhcp-host=d0:3f:27:xx:yy:zz,192.168.20.91

dhcp-host=d0:3f:27:aa:bb:cc,192.168.20.92

# announce the HA machine (192.168.20.1 here as an example) as DNS server and ntp server

dhcp-option=6,192.168.20.1

dhcp-option=option:ntp-server,192.168.20.1

conf-dir=/etc/dnsmasq.d/,*.conf

DNS setup

The dnsmasq will also be serving as DNS server for the cameras. To avoid them to keep querying the hard-coded NTP severs, create a conf file under /etc/dnsmasq.d directory to redirect them to local machine.

address=/time-a.inst.gov/192.168.20.1

address=/time-b.inst.gov/192.168.20.1

address=/time-a-g.inst.gov/192.168.20.1

address=/time-b-g.inst.gov/192.168.20.1

address=/clock.fmt.he.net/192.168.20.1

address=/ntp0.cornell.edu/192.168.20.1

NTPD time server

I like to configure the IP camera to overlay timestamp on the video. Unfortunately the Wyze camera firmware contains a binary that only query several hard-coded NTP servers. But the wz_mini_hacks firmware actually comes with a NTP process to update the time. More on how to set it up in part 2.

For now, make sure the HA machine has NTPD setup properly and ready. Run `sudo ntpq -c lpeer` to check

IP Masquerading

During the initial setup, the Wyze camera needs internet access. To allow the traffic routed to internet and back using NetworkManager and firewalld:

# assuming wlan0 is the device id of the wifi adapter configured with nmcli

sudo nmcli con up wlan0

# move the interface to trusted zone to allow all traffic

sudo firewall-cmd --zone=trusted --change-interface=wlan0

sudo firewall-cmd --zone=trusted --add-masquerade

# assuming eth0 is the LAN interface with internet access

sudo firewall-cmd --direct --add-rule ipv4 nat POSTROUTING 0 -o eth0 -j MASQUERADE

sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i wlan0 -o eth0 -j ACCEPT

sudo firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT

When we are done configuring IP cameras and making sure Frigate can access them, revoke the access and isolate the cameras from internet.

sudo firewall-cmd --direct --remove-rule ipv4 nat POSTROUTING 0 -o eth0 -j MASQUERADE

sudo firewall-cmd --direct --remove-rule ipv4 filter FORWARD 0 -i wlan0 -o eth0 -j ACCEPT

sudo firewall-cmd --direct --remove-rule ipv4 filter FORWARD 0 -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT

sudo firewall-cmd --zone=trusted --remove-masquerade

For a proper setup, should probably also move the wifi adapter to another zone. But well...

That is it for the Linux machine setup. Next part of the notes will focus on setting up the Wyze camera for self-hosting.