Home - joectchan/blog GitHub Wiki
- 2020-04-20 Build MCUBoot. Install signed Zephyr sample USB DFU app in slot 0. Update two signed apps in slot 1.
- 2020-04-20 Installs Zephyr RTOS on Ubuntu on x86. builds Hello World for STM32 Disco-L475-IOT01
- This pdf show implementing USB CDC EEM and Vendor Specific device with Zephyr
- This issue has a comment saying that loopback device and WebUSB device are examples of non-standard class and provide link to documentation on what needs to be changed.
Attended a class taught by Dan Bikle four years ago. website
- AICamp
- Learning NLP
- Natural Language Processing with PyTorch
- Scattertext
- scikit-learn
- spaCy
- Understand LSTM
I have one monitor attached to my Windows laptop. My development are all done on a NUC x86 running Ubuntu. I keep it headless and use RDP from Windows to access Gnome desktop. I also setup VNC access to this Gnome desktop, so that I can choose either RDP or VNC.
This link talks about a lot of customizations I like. I add a few more for my own environment.
I need to enable grub menu on boot because I have Ubuntu 19.10 and 20.04 on my NUC
Use sudo vi /etc/default/grub
to add non-zero timeout.
GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=30
Use sudo update-grub
after finished editing.
Configure ssh server link
Do NOT follow their optional step of allowing ssh login as root!!
Install with sudo apt install clamav rkhunter
Use sudo vi /etc/default/rkhunter
to change these.
CRON_DAILY_RUN="true"
CRON_DB_UPDATE="true"
REPORT_EMAIL="your username on your ubuntu"
APT_AUTOGEN="true"
NICE="19"
Use sudo vi /etc/rkhunter.conf
to change these.
UPDATE_MIRRORS=1
MIRRORS_MODE=0
MAIL-ON-WARNING=your username on your ubuntu
WEB_CMD="curl"
Use sudo vi /etc/rkhunter.conf.local
to add these for specific config for ubuntu and apps that I use, because rkhunter warns against them. I prefer to keep overrides in a separate file, instead of modifying /etc/rkhunter.conf.
PKGMGR=DPKG
ALLOWDEVFILE="/dev/shm/pulse-shm-*"
ALLOWHIDDENDIR=/etc/.java
ALLOWIPCPROC=/usr/bin/gnome-software
ALLOWIPCPROC=/usr/bin/nautilus-desktop
ALLOWIPCPROC=/usr/bin/nautilus
ALLOWIPCPROC=/usr/bin/evince
ALLOWIPCPROC=/usr/lib/firefox/firefox
This is how I started my first scan with rkhunter
sudo rkhunter --update
sudo rkhunter --propupd
sudo rkhunter --versioncheck
sudo rkhunter --check -sk
Besides the warnings that I ended up supressing with rkhunter.conf.local, it gave me a real warning about my poor config of ssd. I need to fix by
/etc/ssh/sshd_config
PermitRootLogin=no
I never need to config clamav. I just run it by clamscan -r -i
I setup cron job to run clamav and rkhunter daily. sudo crontab -e
0 0 * * * /usr/bin/rkhunter -c 2>&1 | mail -s "Report of My Laptop: rkhunter" <your email>
3 0 * * * /usr/bin/clamscan -r -i /root 2>&1 | mail -s "Report of My Laptop: clamscan /root" <your email>
6 0 * * * /usr/bin/clamscan -r -i /home 2>&1 | mail -s "Report of My Laptop: clamscan /home" <your email>
I know rkhunter scan is pretty fast. I don't use root account for anything so clamscan /root should be fast. I estimate they take 3 minutes each. Scan /home takes a long time...
- Config clam This is a much more elaborated configuration of clamav
I have two monitors at work. I use arandr to config their rotation and placement setting. This is very useful when I am not setting up a headless Ubuntu.
sudo apt install arandr
arandr
I setup my headless NUC for RDP to use the built-in rdp client on Windows 10.
- Wifi connection drop will not hurt. Just reconnect.
- RDP desktop size depends on RDP client size when making connection. VNC desktop size does not change.
Make my Ubuntu boots in text mode. Otherwise, my Gnome desktop will be display on the non-existing monitor and xrdp will not be able to show me a desktop remotely.
systemctl set-default multi-user.target
Reboot to confirm Ubuntu boots into text mode. Then install and confirm xrdp runs.
I will visit https://c-nergy.be
and get their xrdp-installer script. I got mine under their Product tab, version 1.2 for ubuntu 20.04. My commands only installs xrdp and check it is running.
(Obsolete commands. Use the script I mentioned instead.)
sudo apt install xrdp
sudo systemctl status xrdp
ip addr show
shows ip address of my NUC. Over on my Windows laptop, I use the built-in rdp client and enter the IP address of my NUC.
hostnamectl status
shows the hostname of my NUC. (which is nuc in my case.) I can do a ping nuc.local
on my Windows laptop to find the ip address of my NUC, thanks to avahi-daemon. link
VNC server is built-in to Ubuntu GUI desktop. If you keep the default graphical.target with systemctl, you can open Settings..Sharing. Enable sharing with the sliding button. Click Screen Sharing. Enable screen share with another sliding button.
Check this setup with command avahi-browse -a
at the shell. You will see.
+ wlo1 IPv6 joseph's remote desktop on nuc VNC Remote Access local
+ wlo1 IPv4 joseph's remote desktop on nuc VNC Remote Access local
When I use another Ubuntu in my LAN to access my NUC machine, I can open Remmina and type nuc.local into the address bar of Remmina and press Enter. The avahi-browse command has already confirmed VNC server is advertising its service in my LAN. Hence, Remmina on another Ubuntu will be able to see that service in the same LAN.
The quality of graphics is poor by default because the VNC connection is using 8 bit color. I will quit and properly add a connection to my NUC. Edit color depth to High color and quality to Medium.
Remmina also support SFTP and SSH protocols. I like to add an sftp connection to NUC to facilitate file transfer between linux machines on my LAN.
Open Terminal and type the following.
sudo apt install gnome-shell-extensions gnome-tweaks
Logout and login. Open Terminal and type the following.
gnome-tweaks
Under General, I turn off animation because I use xrdp to remote access. I need to cut down eye candy.
Under Extensions, I recommend to use Workspace Indicator
. Alternatively, I also like an extension called Workspaces to dock
. link
Allow ssh, rdp and vnc connection within my LAN 192.168.1.X.
sudo ufw allow proto tcp from 192.168.1.0/24 to any port 22,3389,5900
sudo ufw enable
sudo ufw status numbered
I installed KDE on ubuntu on a rpi. It bugs me for root password to do wifi scan all the time. That is because NetworkManager permission is not configured properly.
sudo nmcli general permissions
PERMISSION VALUE
org.freedesktop.NetworkManager.wifi.scan unknown
This command shows the detail of this polkit policy sudo pkaction --action-id org.freedesktop.NetworkManager.wifi.scan --verbose
org.freedesktop.NetworkManager.wifi.scan:
description: Allow control of Wi-Fi scans
message: System policy prevents Wi-Fi scans
vendor: NetworkManager
vendor_url: http://www.gnome.org/projects/NetworkManager
icon: nm-icon
implicit any: auth_admin
implicit inactive: yes
implicit active: yes
I create a new file 46-allow.wifi.scan.pkla
in /etc/polkit-1/localauthority/50-local.d/
to get rid of the permission issue.
[Allow WiFi scan all Users]
Identity=unix-user:*
Action=org.freedesktop.NetworkManager.wifi.scan
ResultAny=yes
ResultInactive=yes
ResultActive=yes
They are installed by default in ~/.local/share/gnome-shell/extensions/
ls ~/.local/share/gnome-shell/extensions/
[email protected]
Enable an extension via command line by
gnome-shell-extension-tool -e [email protected]
This is useful to know for shell scripting some installation.
I come across these pages showing to setup OpenPGP keys on windows. I will try to do that on Ubuntu instead in the future. Generate OpenPGP keys SSH with OpenPGP and YubiKey
When ping nuc.local
does not respond, i.e. mDNS somehow does not work, I use IPv6 instead to connect to my headless linux boxes. First, find the name of the ethernet interface of my local linux box.
ip addr show | grep "state UP"
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
Ssh back to localhost via IPv6
ssh -6 ::1
Show the IPv6 addresses of this interface.
ip -6 addr show dev wlp2s0
or
route -A inet6 | grep wlp9s0
Detect IPv6 neighbors
ip -6 neigh show
Alternatively, detect IPv6-active hosts on a link by using ping6 to the link-local all-node multicast address.
ping6 -I wlp2s0 ff02::1
Ssh to the neighbor, the headless linux box, using its IPv6 address and the local interface used to connect to that address.
ssh -6 fe80::fc42:3cdd:7c2c:57c%wlp2s0
Follow this link to configure Avahi with IPv6
Since I use my old laptop as headless linux box, I want to turn off the LCD screen. Configure using DPMS in text conssole. Login at my laptop into a console. Enter this command to force the LCD screen off.
setterm -blank force -powerdown 1 -powersave powerdown
Link shows the steps to grant myself access to serial port. It is basically adding myself to the dialout
group and reboot.
- Michael Larabel's Phoronix
- Mattias Geniar's Cron Weekly
- i-programmer
- Top 14 Things to do After Installing Ubuntu 20.04 LTS
- Differential Equations as a Neural Network Layers
- We tried to build an end-to-end ML platform. Here’s why it failed
- Python Pitfall: Mutable Default Arguments
- Firmware Update over Bluetooth Low Energy on PineTime Smart Watch
- Configure Mynewt for SPI Flash on PineTime Smart Watch
- MCUBoot Bootloader for PineTime Smart Watch
- git tips Search git for a string. Show file in another branch
- Learn Git Branching
- Read json file from a private github repo
- Understanding CRC
- Pdf of a paper on CRC Polynomial Selection For Embedded Networks
- Interval tree
- log-structured merge-tree
- Parallel Hashmap
- Parallel Hashmap code
- video CppCon 2017: Matt Kulukundis “Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step”
- video CppCon 2019: Matt Kulukundis
- video CppCon 2017: Louis Brandy "Curiously Recurring C++ Bugs"
- Abseil SwissTable
- Abseil Hash
- ControlFlagControlFlag: A Self-supervised Idiosyncratic Pattern Detection System for Software Control Structures
- ThreadSanitizer
- linker 20 part linker essay
- kcachegrind Call graph viewer
- Remake Make debugger
- Makefile flags explanation
- Cross compile with LLVM/Clang
- Zephyr firmware for the nRF52832 PineTime smartwatch
- preprocessor and macro blog page
- c macro C macro metaprogramming
- for each c macro
- c macro for each
- Another c macro trick
- Mentioned in the previous link
- Read this as well
- c wrapper over an API that use JSMN for client-server communcation
- printf %jd %ju
- awesome-cpp
- order c preprocessor library
- chaos c preprocessor library
- b-tree
- hashmap
- nanopb Nanopb is an ANSI-C library for encoding and decoding messages in Google’s Protocol Buffers format with minimal requirements for RAM and code space. It is primarily suitable for 32-bit microcontrollers.
- libraries for protobuf
- GNU poke Analyze binary file
- linkSteps to separate debug symbol out of elf binary
- medium linkCreating and using debug symbol tables with CMake and GDB
- pretty print STL in gdb
- Recipe of config pretty print STL in gdb on ubuntu
- Add python modules to the python interpreter inside gdb
- Config VSCode debug with gdb and pretty print STL
- [Summary of Debugging Tools for Parallel Applications] (http://pramodkumbhar.com/2018/06/summary-of-debugging-tools/)
- Config ddd to use modern gdb and custom pretty printer ddd can record history of a watched variable. Use python script to test variable type and custom pretty print your datatype.
- Gotcha is a library that wraps functions. Tools can use gotcha to install hooks into other libraries, for example putting a wrapper function around libc's malloc.
- GDB dashboard is a standalone .gdbinit file written using the Python API that enables a modular interface showing relevant information about the program being debugged.
- Write up about gdbgui. It can "monitor the value of certain variables over time on a chart." This is the installation guide
- link ARM Micro Trace Buffer and gdb python script to interpret buffer data.
- link Debug ARM hard fault
- link Describe fault handler and PyCortexMDebug gdb py script to parse svd file and apply in gdb
- pdf Tracing Summit 2018 Evolution of self-hosted Tracing with ARM CoreSight
- pdf Hardware Assisted Tracing on ARM with CoreSight and OpenCSD, by Mathieu Poirier
- link Hardware Tracing for Fast and Precise Performance Analysis. Intel and ARM
- link Intel CPU tracing tool vendor books.
- link One blog post from the above vendor. Using the Intel Trace Hub for at-speed printf
- link 2019 Explain Intel Processor Trace
- Linaro OpenCSD lib Decoding CoreSight trace.
- pdf 2014 Hardware-Assisted Software Event Tracing, performance
- link MIPI System Software - Trace (MIPI SyS-TSM) is a common data format for transmitting software trace and debug information between a test system and a device, such as a system-on-chip (SoC) or platform.
- link 2016 Linaro blog. System Trace Module (STM) and its usage. Has link to a sample dtsi file for Spreadtrum’s SC9836
- linkLinux 5.4.47 Documentation/devicetree/bindings/arm/coresight.txt
- link Trace capture/decode with CoreSight driver and Perf on Linux, using arch/arm64/boot/dts/arm/juno-base.dtsi board
- Generate backtrace with abseil-cpp
- Print stack trace example
- Print stack trace example
- Print stack trace example
- backward-cpp C++ header only library to print stacktrace
- Icecream-cpp header only debugging lib for cpp
- Embedded C trace Trace macro. Tool to strip trace text and only transmit binary. Pair with json to reconstruct the message on host.
- PerfettoThe Perfetto Tracing SDK is a C++11 library that allows userspace applications to emit trace events and add more app-specific context to a Perfetto trace.
- Eclipse Trace Compass Visualizing CommonTraceFormat, gdb trace
- uftrace Capture function trace of a user space application compiled with -pg. Slide 5 shows a list of libraries to compile uftrace.
- Linux Full Stack Tracing Cross-compile ARM linux kernel and install Lttng tools. Use shell scripts to start and stop Lttng capture.
- metal-ci Bare metal uniting testing. Python-gdb script helps to print the values on the target when test failed. Or just serial port unit testing, where serial port message shows a test condition failed without providing the runtime value(s) failing that test.
- Always-on Profiling for Production Systems
- AwesomePerfCpp
- Summary of Profiling Tools for Parallel Applications
- Timing + Memory + Hardware Counter Utilities for C / C++ / CUDA / Python
- Performance Application Programming Interface PAPI provides tool designers and application engineers with a consistent interface and methodology for the use of low-level performance counter hardware found across the entire compute system (i.e. CPUs, GPUs, on/off-chip memory, interconnects, I/O system, energy/power, etc.). PAPI enables users to see, in near real time, the relations between software performance and hardware events across the entire computer system.
- USB documentation for software engineer
- USB in a NutShell
- USB CDC-ECM example for STM32F072 microcontroller
- link Forum post about modify ST CDC USB Class code to make a CDC-ECM
- TinyUSB
- Tiny USB ESP32-S2
- IntergatedCircuitsIPoverUSB, USBDevice4Cube, freertos-mcpp, threadx-mcpp
- CherryUSB
- Project Cassini standards-based hardware, boot loader, embedded linux
- Simplifying Embedded Linux with SystemReady IR
- lua-c++
- pikascript PikaScript is an ultra-lightweight Python engine
- c++ embedded Call Stack Logger post, c++17 state machine posts
- F PrimeA C++ framework that provides core capabilities such as message queues and threads
- Evolution of functions in modern cpp
- enum reflection
- Magic enum
- bestofcpp Directory of cpp libraries
- qLibc c library of containers
- ZBar
ZBar is an open source software suite for reading bar codes from various sources, such as video streams, image files and raw intensity sensors. It supports many popular symbologies (types of bar codes) including EAN-13/UPC-A, UPC-E, EAN-8, Code 128, Code 39, Interleaved 2 of 5 and QR Code.
The flexible, layered implementation facilitates bar code scanning and decoding for any application: use it stand-alone with the included GUI and command line programs, easily integrate a bar code scanning widget into your Qt, GTK+ or PyGTK GUI application, leverage one of the script or programming interfaces (Python, Perl, C++) ...all the way down to a streamlined C library suitable for embedded use.
- Kompex SQLite Wrapper for C++
- CBOR
- Embedded Template Library Statically allocated containers
- Heaptrack heap memory profiler for Linux
- Catch2 C++ unit testing framework
- JSON C++ C++11 header only library
- C++ and Embedded Systems
- Bayarea C/C++ User
- Floating point number
- Why would I ever use push_back instead of emplace_back?
- emplace_back
- C++ Core Guidlines
- Guidelines Support Library
- More C++ Idioms
- Nonius c++ header only micro-benchmarking framework
- An Improved Thread with C++20
- c++17 c++20 thread
- c++11 thread
- link ARM and Lock-Free Programming.
C++ code
// Producer thread:
Data_t g_data1, g_data2, g_data3;
std::atomic<bool> g_flag
g_data1 = calc1();
g_data2 = calc2();
g_data3 = calc3();
g_flag.store(true, std::memory_order_release);
// Consumer thread:
if (g_flag.load(std::memory_order_acquire)) {
DoSomething(g_data1, g_data1, g_data2, g_data3);
- Protobuf enum as string
- My own c++ example code showing the result of troubleshooting class member definition of a template class and using std::move() to pass functor into std::thread().
C++ code
Execute online with Coliru
#include <iostream>
#include <utility>
#include <thread>
#include <chrono>
#include <memory>
#include <syncstream>
// TODO: Reference additional headers your program requires here.
/*
Example 1: If a functor has data member of the type std::unique_ptr, then
passing functor object to std::thread() REQUIRES
either std::move(functor object) in order to invoke the default move constructor
or define your copy constructor explicitly
*/
struct Vec3
{
int x, y, z;
// following constructor is no longer needed since C++20
Vec3(int x = 0, int y = 0, int z = 0) noexcept : x(x), y(y), z(z) { }
friend std::ostream& operator<<(std::ostream& os, const Vec3& v) {
return os << "{ x=" << v.x << ", y=" << v.y << ", z=" << v.z << " }";
}
};
class foo
{
public:
foo() { std::cout << "foo()" << " count: " << ++count << " n: " << n << '\n'; };
#if 1
// This functor class does not have a unique_ptr as a data member.
// We can pass it into thread by using:
// std::thread(foo)
// without writing our custom copy constructor. Default copy constructor works.
// You can change the #if to zero to try that.
foo(const foo& org) : n(org.n) { std::cout << "foo(const foo&)" << " count: " << ++count << " n: " << n << '\n'; };
foo(foo&& org) : n(std::exchange(org.n, 0)) { std::cout << "foo(foo&&)" << " count: " << count << " n: " << n << '\n'; };
#else
foo(const foo&) = default;
foo(foo&&) = default;
#endif
void operator()()
{
std::osyncstream synced_out(std::cout);
v1_ = Vec3{3,4,5};
for (int i = 0; i < 5; ++i) {
synced_out << "Thread foo executing\n";
++n;
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
synced_out << "Vec3{3,4,5}: " << v1_ << " n: " << n << '\n';
}
int n = 10;
static int count;
private:
Vec3 v1_;
};
class baz
{
public:
baz() {std::cout << "baz()" << " count: " << ++count << " n: " << n << '\n'; };
#if 1
// This functor class has a unique_ptr as a data member.
// We can pass it into thread by:
// std::thread(std::move(baz))
// Default move constructor works.
//
// Or write and a custom copy constructor that copy everything except the unique_ptr.
// Then we can continue to use the syntax
// std::thread(baz)
baz(const baz& org) : n(org.n) { std::cout << "baz(const baz&)" << " count: " << ++count << " n: " << n << '\n'; };
baz(baz&& org) : n(std::exchange(org.n, 0)), v2_(std::move(org.v2_)) { std::cout << "baz(baz&&)" << " count: " << count << " n: " << n << '\n'; };
#else
baz(const baz&) = default;
baz(baz&&) = default;
#endif
void operator()()
{
std::osyncstream synced_out(std::cout);
v2_ = std::make_unique<Vec3>(0,1,2);
for (int i = 0; i < 5; ++i) {
synced_out << "Thread baz executing\n";
++n;
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
synced_out << "make_unique<Vec3>(0,1,2): " << *v2_ << " n: " << n << '\n';
}
int n = 20;
static int count;
private:
std::unique_ptr<Vec3> v2_ = nullptr;
};
int foo::count = 0;
int baz::count = 0;
/*
Please read main() to understand the point made by example 1. Try uncommenting thread 4 code.
*/
/*
Example 2: Class data member of a template class can only be defined with an initializer.
Without an initializer, that statement is only a declaration statement.
*/
namespace grpc {
class Channel {
public:
Channel(const std::string& target) { std::clog << "Channel to " + target + " | class data member, implicit_init_ptr: " << implicit_init_ptr << '\n'; };
private:
static int* implicit_init_ptr;
};
int global_integer = 42;
#if 1
// Non-template class can define class data member by implicit initialization
int* Channel::implicit_init_ptr;
#else
// Non-template class can also define class data member with explicit initialization
int* Channel::implicit_init_ptr = &global_integer;
#endif
std::shared_ptr<Channel> CreateChannel(const std::string& target) {
return std::make_shared<Channel>(target);
};
} // namespace grpc
namespace ws {
class Op {
public:
Op() { std::clog << "Op()\n"; };
};
class Api {
public:
Api() { std::clog << "Api()\n"; };
};
} // namespace ws
template <typename T>
class WebServiceGrpcClient {
public:
static void ini_channel(const std::string& host) {
WebServiceGrpcClient<T>::channel_ =
::grpc::CreateChannel(host);
};
private:
// All stubs (threads) share a single channel to hwmond.
static std::shared_ptr<::grpc::Channel> channel_;
};
#if 0
// Template class can NOT define class data member by implicit initialization. These lines will be treated as declarations. Compilation will fail.
//
// main.cpp:(.text.startup+0x5b): undefined reference to `WebServiceGrpcClient<ws::Op>::channel_'
// main.cpp:(.text.startup+0x159): undefined reference to `WebServiceGrpcClient<ws::Api>::channel_'
//
template <>
std::shared_ptr<::grpc::Channel> WebServiceGrpcClient<::ws::Op>::channel_;
template <>
std::shared_ptr<::grpc::Channel> WebServiceGrpcClient<::ws::Api>::channel_;
#else
// Template class MUST define class data member with explicit initialization. These lines can compile and run
template <>
std::shared_ptr<::grpc::Channel> WebServiceGrpcClient<::ws::Op>::channel_ = nullptr;
template <>
std::shared_ptr<::grpc::Channel> WebServiceGrpcClient<::ws::Api>::channel_ = nullptr;
#endif
/*
End of example 2
*/
int main()
{
WebServiceGrpcClient<::ws::Op>::ini_channel("op.web.service.com");
WebServiceGrpcClient<::ws::Api>::ini_channel("api.web.service.api");
int n = 0;
foo f;
baz b1;
baz b2;
f.n = 100;
b1.n = 30;
b2.n = 40;
std::thread t1(std::move(f)); // Default move constructor does not destory f.n. Our custom move constructor use std::exchange() and destory f.n after move.
std::thread t2(f); // Use f after std::move() is bad! Doing this just to observe the behavioral difference between the default and our custom move constructor
std::thread t3(std::move(b1)); // // Default move constructor does not destory b1.n. Our custom move constructor use std::exchange() and destory b1.n after move.
std::thread t4(b2); // Compilation error when using the default copy constructor. Unique pointer has no copy constructor. Our custom copy constructor deliberately skips the unique pointer and that make this line work again.
t1.join();
t2.join();
t3.join();
t4.join();
std::cout << "Final value of n is " << n << '\n';
std::cout << "Final value of f.n (foo::n) is " << f.n << '\n'; // Use after std::move() is bad! Doing this for educational purpose.
std::cout << "Final value of b1.n (baz::n) is " << b1.n << '\n'; // Use after std::move() is bad! Doing this for educational purpose.
std::cout << "Final value of b2.n (baz::n) is " << b2.n << '\n';
return 0;
}
- CPM.cmake Setup-free CMake dependency management
- ModernCppStarter A boilerplate template to start new modern C++ project.
- Alternatives to Docker Podman, Buildah, ...
- Learning Go by examples Create a gRPC app in Go
- TinyGo
- TinyGo examples
- MS Beginner's Series to Rust. Video and Github
- lilos: minimum async RTOS in Rust
- Run rust firmware on cortex-m
- Tock OS
- Visual Embedded Rust Programming with Visual Studio Code
- Embedded Rust Book
- Using C Libraries in Rust
- Paper about Rust applications for Zephyr
- Rust on Zephyr RTOS module
- Simple github project showing a Rust app runing on Zephyr
- ITM Using ITM in Rust
- Rux Rux is a hobbyist microkernel written in Rust, featuring a capability-based system similar to seL4.
- Redox Redox is a Unix-like Operating System written in Rust
- Learning resources
- Rust http clients
- Blog post Seem to emphasize on embedded use of rust.
- Support new STM32 Bootstrapping support for the STM32WLE with the Embedded Rust ecosystem
- CXX This library provides a safe mechanism for calling C++ code from Rust and Rust code from C++
- CXX-Qt
- Hubris Hubris is a small operating system intended for deeply-embedded computer systems.
- Bytehound github Bytehound, which is a memory profiler written in Rust
- Python 3 standard library
- cmd Create line-oriented command processors
- prompt_toolkit prompt_toolkit is a library for building powerful interactive command line and terminal applications
- Modern Python Developer's Toolkit
- pipx and visidata
$python3 -m pip install --user pipx
Collecting pipx
Using cached https://files.pythonhosted.org/packages/f6/7c/db71a08fe11bf509bd9fa3b6d89066dc5e8d0623f9280c7c235b4e012103/pipx-0.16.1.0-py3-none-any.whl
Will not install to the user site because it will lack sys.path precedence to colorama in /opt/python3.6c/lib/python3.6/site-packages
$ python3 -m pip install --user --no-deps pipx
Collecting pipx
Using cached https://files.pythonhosted.org/packages/f6/7c/db71a08fe11bf509bd9fa3b6d89066dc5e8d0623f9280c7c235b4e012103/pipx-0.16.1.0-py3-none-any.whl
Installing collected packages: pipx
Successfully installed pipx-0.16.1.0
- microservices with grpc
- Example of pybind11
- Example of Boost.Pythoon
- Kplugs KPlugs is a Linux kernel module which provides an interface for dynamically executing scripts inside the Linux kernel
- Using Python in the Linux Kernel Meetup. Compile micropython to run as linux module
- link Ternminal history of the previous link
- code tracer
- treelib
- treelib to graphviz
- plotly
- monkeyhex
- jupyter Use jupyter to make web app, game
- Python in a container
- MS Learn MS web-based learning. This one is NASA related python learning module.
- link Using Pylance to Write Better Python Inside of Visual Studio Code
- Data Science in Visual Studio Code
- Pygame ebook Learn Python by creating a video game
- Numba Python JIT and parallelize
- fastcore library Python fastcore library
- link Python web frameworks
- Python Exception Class Hierarchy
- Python Package Guide to use Cookiecutter & Poetry to Create a Python Package
- Project template scientific python project template
- Package Python Projects the Proper Way with Poetry shows how to use Poetry with an example and some explanations.
- Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.
- Configuring Python Projects with INI, TOML, YAML, and ENV files
pip install pyyaml
import yaml
try:
with open (path_to_yaml, 'r') as file:
config = yaml.safe_load(file)
except Exception as e:
print('Error reading the config file')
- Read modify write JSON in Python
- The Pandas DataFrame – loading, editing, and viewing data in Python
- Creating Pandas DataFrames from Lists and Dictionaries
- Import csv file into Pandas data frame
- Export Pandas DataFrame to a CSV File
- Filter pandas DataFrame by substring critera
- Example for query/filter Pandas data frame
- Another example for query/filter Pandas data frame
- pandastable example
- pandastable doc for python/tkinter GUI developers who want to include a table in their application that can store and process large amounts of data
- Get parent dir from pathlib.path
One line functions for pdb to save object as a text file using pretty print
def py_pp(o, fname, hdg): from pprint import pprint; outfile = open(fname+'.txt', 'w'); pprint(hdg, stream=outfile); pprint(o, stream=outfile); outfile.close()
def py_pp_vars(o, fname, hdg): from pprint import pprint; outfile = open(fname+'.txt', 'w'); pprint(hdg, stream=outfile); pprint(vars(o), stream=outfile); outfile.close()
pdb can accept multiline command. Use interact
to enter a shell context. You can continue to access the variables you want to examine. In this example, I can print a desc_list
, which contains objects of type fields
. I print the name of the field
object and then the content of the field
object.
interact
from pprint import pprint
for rd in desc_list:
print ('---')
for f in rd.fields:
print (f.name)
print ('---')
pprint (vars(f))
for i in {1..10}; do commandNameHere; done
ps -C node -o pid,rss,pcpu,comm
ps hax -o rss,user | awk '{a[$2]+=$1;}END{for(i in a)print i" "int(a[i]/1024+0.5);}' | sort -rnk2
- link logical op and shell expansion
- link Shell scripting. awk perl array whiptail
- link File browser using bash and whiptail
- link Pure bash script for user to build a menu for user to select
- link Bash cut part of a string
-
link Command grouping with
{cmd1; cmd2 ; cmd3} > single_output.txt
or launch subshell with(cmd1; cmd2 ; cmd3) > single_output.txt
- gRPC-web A JavaScript implementation of gRPC for browser clients.
- Envoy Cloud-native high-performance edge/middle/service proxy
- Learning Go by examples gRPC Create a gRPC app in Go
- microservices with grpc
- rsyslog Setup remote syslog
- rsyslog More examples setting rsyslog
- rsyslog More examples setting rsyslog. Syntax seems to be newer.
- Evolution Set up Evolution to work with MS Office365 account
- LXD container manager
-
Linux homebrew homebrew for linux. Install package for single user without sudo. If your linux system does not have ruby pre-installed. Then use rvm to installl ruby for single user.If your system is old, do not
brew update
. I could get trufflerruby and linuxbrew running on an old pc with fedora 25. I attemptedbrew update
then the newer brew code wants to access ruby in (/usr/bin:/bin:/usr/sbin:/sbin). That defeats my goal of single user install without sudo. At least I have tried. Oh well. - RVM This shows how to install rvm. Basically:
command curl -sSL https://rvm.io/pkuczynski.asc | gpg2 --import -
command curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -
curl -sSL https://get.rvm.io | bash -s -- --version latest
source ~/.rvm/scripts/rvm
rvm list known
# My test pc has an old fedora. It does not have pre-built binaries for ruby 2
rvm install ruby-2.3
Searching for binary rubies, this might take some time.
No binary rubies available for: fedora/25/x86_64/ruby-2.3.8.
Continuing with compilation. Please read 'rvm help mount' to get more information on binary rubies.
Checking requirements for fedora.
Installing requirements for fedora.
Installing required packages: libyaml-devel..josephc password required for 'dnf install -y libyaml-devel':
#I have to use truffleruby
rvm install truffleruby
Checking requirements for fedora.
Requirements installation successful.
Ran into this error when testing PyIDM, a tk and python app:
X Error of failed request: BadLength (poly request too large or internal Xlib length error)
Major opcode of failed request: 139 (RENDER)
Minor opcode of failed request: 20 (RenderAddGlyphs)
Serial number of failed request: 3026
Current serial number in output stream: 3058
Ended up fixing the error by installing font to ubuntu 18.04 and 20.04
sudo apt install fonts-symbola
I had installed quite a number of prerequisites before PyIDM could run. This is my installation steps:
git clone https://github.com/pyIDM/PyIDM.git
sudo apt install ffmpeg fonts-symbola libcurl4-openssl-dev libssl-dev python3-pip python3-pil python3-pil.imagetk python3-tk
python3 setup.py install --user
python3 -m pyidm
- sshfs busy sshfs mounted dir is busy. cannot umount.
- LTTng, Mini Core Dump, Common Trace Format
- Lynis security auditing tool
- graudit application source code static analysis for security problem
- Radare2 binary analysis tool for linux executables.
- Start using systemd as a troubleshooting tool
- Peek is a linux tool that creates silent screencasts of an area of the screen for creating GIF animations or silent WebM or MP4 videos.
- asciinema capture text console and upload the text-based screencast to that website.
- vi
- byobu
- tmux
- screen
- link Start python script in byobu session at startup
- shell history aggregation and search with loki & fsf
- fzf-git integrate git and fzf
- bat replace cat. Has syntax highlight for markdown, source code; git aware; paginate when too large to fit screen
- fd replace find.
- ripgrep replace grep.
- highlight Convert source code to html
- Environment modules
- Docker image. Cmake STM32 project
- Cmake STM32 project
- STM32 Build server
- Build and Test environment based on ubuntu 14.04
- Step-through debugging with no debugger on Cortex-M
- STM32F4 – In-circuit Debugging Revisited
- STM32F4 – Deploy FreeRTOS Embedded OS in under 10 seconds
- STM32F4 – Template Project with Generic Makefile
- STM32F4 – mbed Your Code: Zero Switching Costs When Changing ARM Platforms
- Shell script to generate STM32 project for mbed-RTOS FreeRTOS and SafeRTOS
- STM32F4 – Object-oriented Programming with Embedded Systems
- SWO and ITM debugging/profiling/lofging
- Orbuculum is a set of tools for decoding and presenting output flows from the Debug pins of a CORTEX-M CPU. This link to stackoverflow shows how to use gdb openocd and orbuculum.
- Custom class for STM32 USB device library walk through how to implement non standard USB device with STM32 LL API functions.
- STM32 Greybus
- Blink an LED using Linux, Zephyr & Greybus via Bluetooth
- This project builds an oscilloscope with STM32F030F4 and ST7735 TFT display
- This answer points out STM32 L4 library example code for using i2c repeated start via HAL API
- st-link Open source version of the STMicroelectronics STLINK Tools
- stlink-trace c program that uses libusb to connect to stlink-v2 debugger and access ITM trace port.
- SWO-Parser Python3 script that talks to OpenOCD to get ITM trace.
- QTrace Probe Low cost ETM probe. Client app runs on Windows.
- unbrick jlink probe
- DirtyJTAG
- DirtyJTAGConvert STM32 Bluepill to JTAG adapter
- UrJTAG
- VersaloonJTAG Versaloon firmware for the STM32 Bluepill board
- ThreadX ThreadX is now AzureRTOS
- Visual Studio 2022 + AzureRTOS
- Visual Studio Code + AzureRTOS
- stm32-rsSTM32 Peripheral Access Crates
- Unison RTOS Unison RTOS
- SmoothSmooth provides a set of C++ classes that makes life as a developer easier compared to working directly with ESP-IDF & FreeRTOS APIs.
- Rust on the ESP32
- esp-idf-template
- esp-whoimage processing dev platform
- List of edge AI links
- AI quora post
- Practical Deep Learning for Coders
- AICamp has some free AI webinars
- Kaggle
- NanoEdge AIruns unsupervised learning and inferences on one MCU.
- 2020 22 tools for ML
- 2020 Sept GPU buying
- multi core ML in Python
- BERT BERT and Google search
- GFPGAN Face restoration for blurry picture
- Gradio app Framework to wrtie in Python some web apps to demo your machine learning models
- buildroot example Buildroot adding new package example
- How to cross-compile for embedded with CMake like a champ
- Better Firmware with LLVM/Clang
- LLVM Embedded Toolchain for Arm
- GNU Toolchain for the Cortex-A Family
- Backblaze Hard Drive Stats Q1 2020
- Chip Design with Deep Reinforcement Learning
- Adaptive FPGA Placement Optimization via Reinforcement Learning
- SystemRDL compiler
- ZenUML Online UML tool whose language syntax is similar to c++/java
- Mermaid Online UML tool whose language syntax is similar to markdown
- Simple intro to graphviz language
- online graphviz 1 Good gallery
- online graphviz 2 Export different output format
- online graphviz 3 Zoom in/out preview. Undo/redo edit view.
- link Text to UML tools – Fastest way to create your models
- Gaphor Desktop UML graphical editor
- link PlantUML with VSCode
- link PlantUML Hitchhikers Guide
- link Support UML diagrams in markdown with plantuml syntax
- link Using plantuml in AsciiDoc on GitHub
- Borwser extension PlantUML Visualizer extension
- link Use PlantUML proxy service to render UML for github page
- link How to integrate UML diagrams into GitLab or GitHub
- link A Dramatic Tour through Python’s Data Visualization Landscape (including ggplot and Altair)
- Altair
- Plotly Dash
- ETW/xperf profiling
- Windows hdd as VM Configured a VirtualBox VM to use a physical Windows drive on my Linux workstation
- /etc/wsl.conf Windows path mixing with linux path caused a lot of troubles for me. Need to disable windows path inside linux. Another link to a description of what wsl.conf contains.
- WSL documentation I used their tutorial to install sqlite3 and DirectML
- Miniconda The instructions for DirectML setup include setting up development environment using miniconda.
- Connecting USB devices to WSL
- Use Intel iGPU in WSL for OpenVINO from MS
- Use Intel iGPU in WSL for OpenVINO from Intel
- Use yubikey from WSL I have NOT tested this method!
- Tails Tails is a portable operating system that protects against surveillance and censorship.
- Flowcrypt FlowCrypt is the simplest PGP encryption for Gmail
- Keybase Keybase is a safe, secure, and private app for everything you do online.
- OpenKeychain
- Oversec Overlay encryption over Android apps. Symmetric keys based on passphrase
- pass the standard unix password manager
- Yubikey Guide
- Yubikey SSH
- Generate ssh key for a hadware security key
- Github Host key
- link Switch from OpenSSH ssh-agent to GnuPG as ssh-agent
- Embedded Interview Topics Many RTOS tutorials too.
- Programmer interview test
- Interview Handbook
- Levels.fyi
- Layoffs.fyi
- Leetcode
fmtlibUse fmtlib as header only library
- Installing loguru
- Another blog about loguru
- fmtlog
- G3log
- One of two Nanolog
- Nanolog, by completely different author
- binlog
- Visidata This is a ncurse type python program. It reads csv, json, ... etc and present a grid of data (a sheet). You can select rows based on some criteria and spawn a new sheet. You can define new columns using Python expression. You can join two sheets.
- q - Text as Data Run SQL directly on csv files
- Credit freeze; Opt out credit increase offer; Annual credit report Link
- Wikipedia explanation It says since 2016 Linux uses rst for documentation.
- Restructured Text and Sphinx CheatSheet
- Its homepage Go to the "Try AsciiDoc on the Web" section of the homepage for online tools to try (AsciiDocBox and DocGist)
- An opinion why use AsciiDoc
- How to Deploy a GatsbyJS Blog on GitHub Pages as a User Site
- How Gatsby Works with GitHub Pages
- Markdown Blog Starter
- [Gatsby Pocket] (https://jamstackthemes.dev/theme/gatsby-theme-pocket/)
I spent an entire day trying out many themes. I ended up using Minimal Mistakes Other themes gave me too many problems. I had to delete my https://github.com/joectchan/joectchan.github.io
repo to start from scratch.
Use this to clone mmistakes as my new https://github.com/joectchan/joectchan.github.io
repo. Use this to create a favicon.ico and save it at the root of my repo because mmistakes does NOT provide a favicon.ico.
I ignored the entire GETTING STARTED section and plunged into CUSTOMIZATION.
- Online md editor with preview Dillinger
- Free or paid themes Jekyll Themes
- Free videos and pictures for building website. Pexels
- Build browser based presentation (https://github.com/hakimel/reveal.js)
- Nice clean theme with text Ed
- Pictures and background video theme Bef
- Clone of Medium Memoirs
- Dev oriented theme Anon Developer
- Modernized with html5 theme Forty
- Feature rich theme Feeling Responsive
- Calendar based theme (http://jlord.us/sheetsee-calendar/)
- Map based theme (http://jlord.us/hack-spots/)
-
link Add JavaScript code in .md file for GitHub Pages. However, I was able to embed a mermaid graph in my github page. That involved using
mermaid.min.js
and the code did NOT show. I had to explicitly add another paragraph with a code block to explicitly show the code. May be the link referred to some point in history when github page did not hide script. Saved here just for reference. -
HTML Tags You Can Use on GitHub
<details> <summary></summary></details>
is very useful
-
Jitsi WebRTC base video conference. Just use browser on computer. No download no account is needed. (Android and iOS needs to download mobile app, however.
-
Netgames Love letter, Avalon, Secret Hitler, ... Just use browser on mobile phone. No download no account is needed.
-
Maque 4 player mahjong. Can either play as guest or register for a free account.
-
List of online games