3 ‐ Development - davisssamuel/notis GitHub Wiki

This section outlines the development process. It includes guidelines on how to contribute, the code of conduct, information on reporting issues, and details about the overall development workflow including testing and debugging. This section helps ensure a smooth collaboration process among contributors.

nostr-tools module

Notis makes use of a very helpful react module called nostr-tools , which provides a number of useful functions that allow much easier implementation of basic nostr client functionality

Install nostr-tools using:

bun -g install nostr-tools

Choosing a Pool of Relays

import { SimplePool } from 'nostr-tools/pool'

const pool = new SimplePool()

let relays = ['relay1', 'relay2', 'relay3']

Publishing to relays is changed often so navigating to the Github is best for understanding.

Decoding Keys using nostr-tools/nip19

To actually use the Nostr protocol to publish and sign notes, keys must be formatted properly. However, the Bech32 encoding that NIP-19 uses is not the proper format for use in relays. Relays require the hex string of keys. You can convert from Bech32 to Hex using:

import { nip19 } from 'nostr-tools'
let bechKey = "npub1sqmqhhuhqq7pqlk20kl2nw520nt7h9jlr6ddp27ynmpvuyt7d4uq54lfxp";
let keyObj = nip19.decode(bechKey);
console.log(keyObj)

If the code was successful, it should output the following:

{
    "type": "npub",
    "data": "80360bdf97003c107eca7dbea9ba8a7cd7eb965f1e9ad0abc49ec2ce117e6d78"
}

To view the hex version of the key you need to access the data portion of the object using:

import { nip19 } from 'nostr-tools'
let bechKey = "npub1sqmqhhuhqq7pqlk20kl2nw520nt7h9jlr6ddp27ynmpvuyt7d4uq54lfxp";
let keyObj = nip19.decode(bechKey);
console.log(keyObj.data)

If the code was successful, it should output: 80360bdf97003c107eca7dbea9ba8a7cd7eb965f1e9ad0abc49ec2ce117e6d78

This method works for all of the following key types: nprofile, nevent, naddr, nrelay, nsec, npub, note

NIP-04 Deprecation [December 2023]

In December 2023, NIP-04 was deprecated in favor of NIP-44 v2 for encryption and decryption of notes and messages. This is due to NIP-04 not being a state of the art as it leaks metadata meaning private messages are not really private. NIP-44 v2 uses the following steps for encryption:

  1. Calculate conversion key
  2. Generate a random 32-byte nonce
  3. Calculate message keys
  4. Add padding
  5. Calculate MAC
  6. Base64 encode

NIP-44 v2 uses the following steps for decryption:

  1. Check the first character of the payload
  2. Decode base64
  3. Calculate conversion keys
  4. Calculate message keys
  5. Calculate MAC
  6. Decrypt ciphertext
  7. Remove padding

Switching from NIP-04 to NIP-44 can be done easily by using the nostr-tools implementation of NIP-44 and replacing all uses of NIP-04 with nostr-tools NIP-44 in your project.

@nostr-dev-kit/ndk module

Notis makes use of another very helpful react module called @nostr-dev-kit/ndk module , which provides a number of useful functions that allow much easier implementation of basic nostr client functionality

Install @nostr-dev-kit/ndk module using:

bun -g install @nostr-dev-kit/ndk

Choosing a pool of relays

In order for NDK for function properly within react, a pool of relays must be specified. To do so, paste the following code:

const ndk = new NDK({
	explicitRelayUrls: [
		"wss://relay.damus.io",
		"wss://atlas.nostr.land",
		"wss://ca.purplerelay.com",
		"wss://northamerica-northeast1.purplerelay.com"],
});

Once this runs, a new NDK instance is created and ready to be used. Before being able to do anything with the relays, you must connect to them. Do so by using the following code:

await ndk.connect();

Querying for profile data

To query the pool of selected relays for profile data, you will be using the ndk object you created above. To query for data, paste the following code:

const usr = ndk.getUser({
	npub: "npub1l9mg709fnx7pvcd0c29zdkffr3v4uu6re4tnp7x7vt9dsr580ffqlhk4j2"
})

In this case, just replace the npub parameter with the public key of the profile you want to retrieve data for. This usr object now contains functions that can be called to perform certain actions for that user. To fetch profile data:

usr.fetchProfile().then((profile) => {
	console.log(profile);
});

This profile object contains various pieces of data. For example:

{
    "name": "Notis Dev Test Profile",
    "image": "https://void.cat/d/GQmo1W7DN9Mb5RcRkrFWvb.webp",
    "about": "Test profile for the notis dev team"
}

Querying for Message data

async function queryMessages(pubkey, myPubKey, func, eose) {

const ndk = new NDK({
	explicitRelayUrls: getRelays()
})
await ndk.connect()

ndk.subscribe([{
	kinds: [4],
	authors: [myPubKey],
	},{
	kinds: [4],
	authors: [pubkey],
}]).on("event", async (e) => {
	if (((e.pubkey == myPubKey && e.tags[0][1] == pubkey) ||
		(e.pubkey == pubkey && e.tags[0][1] == myPubKey)) &&
		e.kind == 4)
	{
		func(e)
	}
}).on("eose", (e) => {
	eose(e)
})}

To query for message data between two users, we need to create two new listeners: one to listen for messages I sent to THEM:

kinds: [4],
authors: [myPubKey],

and for messages that THEY sent to ME:

kinds: [4],
authors: [pubkey],

Once we receive those messages, we need to have some extra filters in place to ensure no additional erroneous messages were returned. At that point, we can call the function passed in the parameters to handle the message event.

if (((e.pubkey == myPubKey && e.tags[0][1] == pubkey) ||
		(e.pubkey == pubkey && e.tags[0][1] == myPubKey)) &&
		e.kind == 4)
	{
		func(e)
	}

Querying for Contact data

async function queryContacts(func) {
	const ndk = new NDK({
		explicitRelayUrls: getRelays()
	})
	await ndk.connect();

ndk.subscribe({
	kinds:[3],
	authors:[await getPublicKeyHex()]
}).on("event", (e) => {
	func(e)
})}

We only need one listener to query for contact information: Kind 3 events published by me:

kinds:[3],
authors:[await getPublicKeyHex()]

This automatically gives us only the most recent event to ensure we have the most up-to-date contacts list.

nostr-relaypool-ts

A Nostr RelayPool implementation in TypeScript with the goal being a simpler implementation than the nostr-tools simple pool. Install in your project with:

npm i nostr-relaypool

The usage is similar to the nostr-tools simple pool but makes publishing to multiple relays easier. Publishing to relays is changed often (as we are early in the development of the Nostr protocol) so navigating to Github is best for understanding.

import {RelayPool} from "nostr-relaypool";

let relays = [
  "wss://relay.damus.io",
  "wss://nostr.fmt.wiz.biz",
  "wss://nostr.bongbong.com",
];

let relayPool = new RelayPool(relays);

We no longer use this, as we discovered ndk was able to handle multiple relays better than this library was able to

nostr.watch API

nostr.watch provides an API that can be used for choosing default relays to publish to. This can be helpful for publishing to relays since relays go offline frequently and this will keep an up-to-date list of online relays.

Setting up the Proxy Server

Install NGINX using:

sudo apt install nginx

Using systemctl to run the React Native project

In order for a proxy server to effectively work, there must always be an instance of your Bun app running in the background. This can be done using .service files and systemctl on Ubuntu 22.

We used expo to render our app for the web, and the command to run the web app using bun is

bun run web

Then, create a script called run.sh

nano run.sh

Paste the following code into the .sh script file you just created:

#! /bin/bash
source ${HOME}/.bashrc

cd <path>/<to>/<project>/<directory>
bun run web

Next, create a .service file to always have the Bun app using:

sudo nano /lib/systemd/system/[service name].service

NOTE: You can replace [service_name] with whatever your service will be called. For this example, we called it notis

Paste the following code into the .service file you just created:

[Unit]
Description=Notis Development Webserver
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=notis
ExecStart=<path>/<to>/<script>/run.sh

[Install]
WantedBy=multi-user.target

NOTE: User may have to be changed to reflect the user that the Bun application is running on.

Additionally, ExecStart may need to be changed based on where your script is located. For this example, our run.sh script is located in /home/notis/Code/notis/code/scripts/run.sh

Start the service using:

sudo systemctl start [service_name] && systemctl enable [service_name]

Now, check to ensure the service is running using:

sudo systemctl status [service_name]

If you were successful in the previous steps, it should output the following:

● notis.service - Notis Development Webserver
     Loaded: loaded (/lib/systemd/system/notis.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-20 16:30:56 EDT; 11min ago
   Main PID: 33398 (run.sh)
      Tasks: 44 (limit: 19089)
     Memory: 341.5M
        CPU: 14.416s
     CGroup: /system.slice/notis.service
             ├─33398 /bin/bash /home/notis/Code/notis/code/scripts/run.sh
             ├─33399 "npm start" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""
             ├─33415 sh -c "react-scripts start"
             ├─33416 node /home/notis/Code/notis/code/frontend/node_modules/.bin/react-scripts start
             └─33427 /usr/local/bin/node /home/notis/Code/notis/code/frontend/node_modules/react-scripts/scripts/start.js

NOTE: If you notice the webserver is displaying outdated information or elements, you can restart the server using:

sudo systemctl restart [service_name]

Using systemctl to setup a NGINX proxy server

Assuming NGINX is installed, create the following configuration file:

sudo nano /etc/nginx/conf.d/[service_name].conf

Paste the following lines into the .conf file you just created:

server {
    location / {
        proxy_pass http://localhost:19006/;
    }
}

Note that 19006 is the default port expo uses for localhost

Test the configuration file with:

sudo nginx -t

If you were successful in the previous steps, it should output the following:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Just as before, start the proxy server using:

sudo systemctl start nginx && systemctl enable nginx

Now, check to ensure the service is running using:

sudo systemctl status nginx

If you were successful in the previous steps, it should output the following:

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-20 14:12:58 EDT; 17min ago
       Docs: man:nginx(8)
    Process: 19201 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, s>
    Process: 19202 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/S>
   Main PID: 19203 (nginx)
      Tasks: 17 (limit: 19089)
     Memory: 13.6M
        CPU: 50ms
     CGroup: /system.slice/nginx.service

Now, find the IP address of your machine and visit:

http://[your_ip_address]

Relay Implementation

Old

First, clone the nostr-rs-relay into the VM using:

git clone -q https://git.sr.ht/\~gheartsfield/nostr-rs-relay && cd nostr-rs-relay

Second, we need to setup a container using the Dockerfile which can be done using:

docker build -t nostr-rs-relay .

The nostr-rs-relay uses a SQLite database. Create a directory for the database using:

mkdir data

Finally, the relay can be configured and ran using:

sudo docker run -it -p 7000:8080 \ 
--mount src=$(pwd)/config.toml,target=/usr/src/app/config.toml,type=bind \  
--mount src=$(pwd)/data,target=/usr/src/app/db,type=bind \ 
nostr-rs-relay:latest

Docker should be setup and running properly.

Useful Docker Commands

Description Command
Check running containers sudo docker ps
Check all available containers sudo docker ps -a
Check the logs of a container sudo docker logs [container_id]
Start a container sudo docker start [container_id]
Stop a container sudo docker stop [container_id]
Remove a container (the container must not be in use) sudo docker rm [container_id]

SQLite Installation

The nostr-rs-relay uses SQLite to store events. Install SQLite using:

sudo apt install sqlite3

Querying the nostr-rs-relay Database

To query the database, go to the directory where the .db output file is located. You can see everything in the event table using:

sqlite3 nostr.db 'SELECT * FROM event'

Issues with this implementation

We were having isseus connecting to this relay from within our app. Nostr-tools and NDK did not seem to like it. We decided to move on from this implementation for a while. Once we began running into issues with the public relays we were using, we decided to try again to make our own relay using Umbrel.

New (Umbrel)

We used a relatively new set of integrated software packages called Umbrel. This allowed us to easily set up and install a nostr relay without any additional overhead. After installing Umbrel using the instructions on their site, we installed the relay application. It was up and running in no time. Using this relay was as simple as vusing ws://< machine ip >: 4848.


Relay Testing

NOSCL Installation

NOSCL is a command-line NOSTR client that will allow for the relay to be tested easily. GO is necessary to use NOSCL. Install GO using:

sudo apt install golang -y

Now, install NOSCL using:

go install github.com/fiatjaf/noscl@latest

NOSCL Usage

First, navigate to your GO binary folder using cd go/bin

Now, a relay has to be added to the NOSCL application. The example below adds a local host relay running on port 7000:

./noscl relay add ws://relay.url

Next, a private key needs to be generated to allow for messages to be sent to relays. This can be done using:

./noscl key-gen

Now that a private key has been generated, set it as the usable NOSCL private key using:

./noscl setprivate <hex private key>

Everything is now set up. A simple message can be published by using:

./noscl publish "hello world"

APIs

QR Code Generation

To generate QR codes for adding friends in nostr, we are using a free public QR code generating API called api.qrserver.com. To generate QR codes using this API, simply visit the following link in your code:

https://api.qrserver.com/<most recent version>/create-qr-code/?size=[width]x[height]&data=[data]

Replace the [width] and [height] parameters with integer values for the width and height of the QR code image you want. Replace the [data] parameter with whatever data you want to embed into the QR code. The API will return an image file for the generated QR code, which can be embedded into the src attribute of an img tag like so:

<img src={
	"https://api.qrserver.com/v1/create-qr-code/?size=512x512&data=" + 
	"This is a test"
	}></img>

This img will render a QR code of size 512x512 with the embedded data being This is a test

Random Profile Picture Generation

To generate random profile pictures for users without a profile picture added, we are using a free public API called api.dicebar.com. Simply visit the following link in your code:

https://api.dicebear.com/<most recent version>.x/identicon/svg?seed=[seed]

Replace the [seed] parameter with whatever seed you want to use to generate the random image. In our case, we made the seed the user's public key so it was consistently the same if they did not have a profile picture added.

The API will return an image file for the generated random picture, which can be embedded into the src attribute of an img tag like so:

<img className="profile-image" src={'https://api.dicebear.com/5.x/identicon/svg?seed=' + Math.random()}></img>

React Native

Setup

We use Expo to streamline the development using React Native for our app. In order to use Expo we must first have Node.js installed (the LTS version). Then install Expo using

npx expo

React Native Error Handling

check the node.js version by using

node -v

If the currently used version is below version 16 then the project will not run. One way to fix this is to install a manager using

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash

Once the manager is installed switch to a newer node.js version by using

nvm install 20

If bun run web does not work for the project then it needs to be run using npx expo. Before being able to run the program run

npm install

Once that finishes, run

sudo npm i undici -g

Now, the project should be setup and all the necessary packages are installed. Run the project using

npx expo --web -c

Outstanding Issue: React Native to React Native Web

React Native Web is designed to bridge the gap between React Native and React with all the components of React Native being implemented into React Native Web. React Native Web is intended to make porting mobile applications made with React Native to the web easier. However, as of writing this (9 April 2024), React Native Web is not complete enough to be used over React or even to easily switch over from React Native. If React Native Web is completed, these are the steps that can be taken to switch from React Native to React Native Web. First, create a new React Native project with:

npx react-native init DemoProject

Then, create all the necessary folders and files from the previous project including index.html and index.js. Next, install the dependencies with:

yarn add react-dom react-native-web
yarn add --dev react-scripts

Then, add the following snippets to the named files: index.html

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width">
        <title> Demo Project </title>
    </head>
    <body>
        <div id="root"></div>
    </body>
</html>

index.native.js

import {AppRegistry} from 'react-native';
import App from './src/App';
import {name as appName} from './src/app.json';

AppRegistry.registerComponent(appName, () => App);

index.js

import {AppRegistry} from 'react-native';
import App from './App';
 
AppRegistry.registerComponent('App', () => App);
AppRegistry.runApplication('App', {rootTag: document.getElementById('root')});

App.js

import React from 'react';
import {SafeAreaView, StatusBar, Text, View} from 'react-native';
 
const App = () => {
 return (
   <SafeAreaView>
     <StatusBar />
     <View>
       <Text style={{fontSize: 24}}>Hello World</Text>
     </View>
   </SafeAreaView>
 );
};
 
export default App;

Change the package.json scripts section to include:

"web": "react-scripts start"

After completing these steps, the application is ready for the web. However, as previously stated this does not currently work very well as many features are not ported to React Native Web, and depending on the size on the size of the project switching over packages and making corrections to certain libraries could be a lot more work than using a different framework.

Roadblock: Crypto dependencies on React Native Mobile

We consistently ran into issues with the crypto library on react native. This kept us from being able to actually develop a mobile app that was able to excrypt and decrypt messages.

Some issues we ran into along the way:

Text Decoder and Text Encoder

If this is an issue where you try to run the app and get a TextEncoder or TextDecoer does not exist error, then it is most likely the nostr-tools module not working properly. For now, it is probably a bug that is yet to be fixed in the nostr-tools code. To fix it manually:

  • go to the file URL specified by the error. If there is no URL provided, it is most likely: node_modules/nostr-tools/lib/cjs/index.js
  • add import { TextEncoder, TextDecoder } from "text-encoding"; to the very top

Your issue should be solved now.

Polyfilling the Crypto.js Library

Even after following several online tutorials and spending weeks attempting to solve issues with polyfilling, we were unable to find resolution. It did not seem like it was going to be as much of an issue as it was, since plenty of stack overflow posts and other wikis provided different solutions to our exact issue. Nothing seemed to work, however, which ended dup setting us back quite far. Ultimately, we settled on rendering the mobile application on the web in order to gain access to the crypto library. Once we made this decision, we were able to move forward with more efficient development.

⚠️ **GitHub.com Fallback** ⚠️