For feedback on this guide, please visit this issue.

Any found issues in this release, please visit this issue.

This document outlines some of the upcoming Bitcoin Core 32.0 release changes and provides steps to help test them. This guide is meant to be the starting point for experimentation and further testing, but is in no way comprehensive! After running through the steps in this guide, you are encouraged to do your own testing. To do additional tests can be as simple as testing the same features in this guide but trying it a different way. Even better, think of features you use regularly and test that they still work as expected in the release candidate. You can also read the release notes to find something not covered in this guide. This is a great way to be involved with Bitcoin's development and helps keep Bitcoin running smoothly and bug-minimized!

Your help in this endeavor is greatly appreciated.

The Bitcoin Core 32.0 release introduces a range of updates.

The guide describes how to test the following updates:

• 1.1 new rpc: exportwatchonlywallet
• 1.2 new rpc: exportasmap new rpc
• 1.3 new rpc: getopenrpcinfo and rpcdiscover
• 2.1 updated rpc: getrawtransaction mempool-rpcs-getrawmempool getmempoolentry testmempoolaccept submitpackage
• 2.2 updated rpc: psbt v2 default
• 2.3 updated rpc: nonmempool field in getbalances
• 2.4 updated rpc: dumptxoutset to a named pipe
• 2.5 estimatesmartfee combined estimator
• 3.1 wallet : addhdkey derivehdkey
• 4.1 REST API: HTTP server rewrite

This chapter outlines the steps to configure a proper testing environment. Skip to the tests

Choose a directory to work from. Set the RC_TEST_DIR environment variable to the chosen directory for testing, e.g., export RC_TEST_DIR=~/rctesting. This designates a workspace/folder in your home directory (~) named rctesting for test-related files.

export RC_TEST_DIR=~/rctesting
mkdir $RC_TEST_DIR
cd $RC_TEST_DIR

Current Release Candidate: Bitcoin Core 32.0rc2 (release-notes)

There are two ways to grab the latest release candidate: from source code or pre-compiled binary. Choose one of the two.

The source code for the release candidate can be obtained from the releases page, or by using git directly:

cd $RC_TEST_DIR
git clone https://github.com/bitcoin/bitcoin.git -b v32.0rc2 --depth 1
cd bitcoin

(continue to step 3 compile)

If using a pre-compiled binary is preferred, make sure to obtain the correct one for your systems architecture. When using the pre-compiled binary, extract the package to the folder set in RC_TEST_DIR. (example for x86_64-linux):

cd $RC_TEST_DIR
wget https://bitcoincore.org/bin/bitcoin-core-32.0/test.rc2/bitcoin-32.0rc2-x86_64-linux-gnu.tar.gz
tar xf bitcoin-32.0rc2-x86_64-linux-gnu.tar.gz

(continue on step 4)

Before compiling, make sure that your system has all the correct dependencies installed.

Compiling from source is highly dependent on your systems architecture and operating system. Refer to the Bitcoin Core compilation guide for details;

• UNIX/Linux
• macOS
• Windows
• FreeBSD
• NetBSD
• OpenBSD

Some testing in this guide involves using previous releases such as v31.1.

Obtain the v31.1 binary from here and extract it to the folder set in RC_TEST_DIR, (example for v31.1 x86_64-linux):

cd $RC_TEST_DIR
wget https://bitcoincore.org/bin/bitcoin-core-31.1/bitcoin-31.1-x86_64-linux-gnu.tar.gz
tar xf bitcoin-31.1-x86_64-linux-gnu.tar.gz

Create temporary data directories for each version.

export DATA_DIR_32=/tmp/32-rc-test
mkdir $DATA_DIR_32
export DATA_DIR_31=/tmp/31-test
mkdir $DATA_DIR_31

Specify path for executable binary, which depend on your selection in step 2: compiling from source or using a pre-compiled binary.

If testing with v32 from compiled source, set:

export BINARY_PATH_32=$RC_TEST_DIR/bitcoin/build/bin

If testing from v32 downloaded binaries, set:

export BINARY_PATH_32=$RC_TEST_DIR/bitcoin-32.0rc2/bin

Specify path for the executable binary from the previous release, e.g. for v31.1:

export BINARY_PATH_31=$RC_TEST_DIR/bitcoin-31.1/bin

To avoid specifying the data directory (-datadir=$DATA_DIR) on each command, create these aliases:

alias bitcoind32="$BINARY_PATH_32/bitcoind -datadir=$DATA_DIR_32"
alias bcli32="$BINARY_PATH_32/bitcoin-cli -datadir=$DATA_DIR_32"

alias bitcoind31="$BINARY_PATH_31/bitcoind -datadir=$DATA_DIR_31"
alias bcli31="$BINARY_PATH_31/bitcoin-cli -datadir=$DATA_DIR_31"

The alias datadir-cleanup is created to make resetting the data directory easier between tests.

alias datadir-cleanup='rm -rf $DATA_DIR_32; mkdir -p $DATA_DIR_32; rm -rf $DATA_DIR_31; mkdir -p $DATA_DIR_31;'

Verify that all versions are correct (e.g. that there are no accidental duplicates):

bcli32 --version | grep version
bitcoind32 --version | grep version
bcli31 --version | grep version
bitcoind31 --version | grep version

Expected results:

Bitcoin Core RPC client version v32.0.0rc2
Bitcoin Core daemon version v32.0.0rc2 bitcoind
Bitcoin Core RPC client version v31.1.0
Bitcoin Core daemon version v31.1.0 bitcoind

If the output of the commands is as expected, the testing environment is set up correctly to continue with this testing guide!

The new exportwatchonlywallet RPC creates a watchonly wallet file from an existing descriptor wallet. The exported file contains the wallet's public descriptors (with derived key caches where needed), transactions, and address book data, but no private keys. It can be imported on another node using the existing restorewallet RPC. #32489

Start Bitcoin Core 32.0 in regtest:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon

Generate a wallet, address, and mine 101 blocks:

bcli32 createwallet "satoshi"
bcli32 generatetoaddress 101 $(bcli32 getnewaddress)

Now call getwalletinfo to see the contents of the wallet.

bcli32 getwalletinfo

The response should be something like this:

{
  "walletname": "satoshi",
  "walletversion": 169900,
  "format": "sqlite",
  "txcount": 101,
  "keypoolsize": 4000,
  "keypoolsize_hd_internal": 4000,
  "private_keys_enabled": true,
  "avoid_reuse": false,
  "scanning": false,
  "descriptors": true,
  "external_signer": false,
  "blank": false,
  "birthtime": 1789496642,
  "flags": [
    "last_hardened_xpub_cached",
    "descriptor_wallet"
  ],
  "lastprocessedblock": {
    "hash": "61d5df7da84304752a309dd83647f1380d998d0003dfab33b37bcff2b5d79f66",
    "height": 101
  }
}

Now export a watch-only wallet from satoshi:

bcli32 -rpcwallet=satoshi exportwatchonlywallet $DATA_DIR_32/watchonly.dat

Stop the node:

bcli32 stop

Start the old Bitcoin Core node in regtest:

echo "regtest=1" > $DATA_DIR_31/bitcoin.conf
bitcoind31 -daemon

Import the watchonly wallet:

bcli31 restorewallet "satoshi" $DATA_DIR_32/watchonly.dat

Now call getwalletinfo on the restored wallet to confirm it holds no private keys:

bcli31 getwalletinfo | jq '{private_keys_enabled: .private_keys_enabled,txcount: .txcount }'

The response should show private_keys_enabled as false, and txcount matching the original wallet's 101 transactions:

{
  "private_keys_enabled": false,
  "txcount": 101
}

Stop the current version node (v31) and clean up:

bcli31 stop && sleep 5  &&  datadir-cleanup

A new exportasmap RPC writes the ASMap data embedded at build time to a file. #33920

Start Bitcoin Core 32.0 in regtest:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon

Export the embedded ASMap:

bcli32 exportasmap $DATA_DIR_32/asmap.dat

Expected output:

{
  "path": "/path/to/DATA_DIR_32/asmap.dat",
  "bytes_written": 1582208,
  "file_hash": "03580ade8ec0036ad3d6a5a91602b995c89387f722d3fee58127219de6aafc12"
}

Check that the file exists and has the same size as the bytes_written output from previous RPC command:

ls -l $DATA_DIR_32/asmap.dat

Check that the exported file can be loaded. Restart the node with the file as -asmap:

bcli32 stop && sleep 5
bitcoind32 -daemon -asmap=$DATA_DIR_32/asmap.dat
sleep 5
grep -i asmap $DATA_DIR_32/regtest/debug.log

The log should show the file being opened and a line like:

Using asmap version <hash> for IP bucketing

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

A new getopenrpcinfo RPC returns a machine-readable OpenRPC 1.4.1 description of the RPCs available on the running node, including their parameters and result schemas. The new rpc.discover method provides the same public document using OpenRPC’s standard service discovery method. To include hidden commands and arguments, call getopenrpcinfo with show_hidden=true. #34683

Start Bitcoin Core 32.0 in regtest:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon

Check the OpenRPC version of the document:

bcli32 getopenrpcinfo | jq .openrpc

Expected output:

"1.4.1"

Show the openrpc info block:

bcli32 getopenrpcinfo | jq .info

Output:

{
  "title": "Bitcoin Core JSON-RPC",
  "version": "v32.0.0rc2",
  "description": "Autogenerated from Bitcoin Core RPC metadata."
}

getopenrpcinfo should list every RPC shown by help. Compare the two counts:

bcli32 getopenrpcinfo | jq '.methods | length'
bcli32 help | grep -c '^[a-z]'

Output:

156
156

The new RPCs from this release should be included:

bcli32 getopenrpcinfo | jq '[.methods[].name] | map(select(. == "exportwatchonlywallet" or . == "addhdkey" or . == "derivehdkey" or . == "exportasmap"))'

Output:

[
  "addhdkey",
  "derivehdkey",
  "exportasmap",
  "exportwatchonlywallet"
]

rpc.discover should return the same output as getopenrpcinfo:

diff <(bcli32 getopenrpcinfo | jq -S .) <(bcli32 rpc.discover | jq -S .)

Expected output:

No output expected because they are equal

Since rpc.discover is meant for generic OpenRPC clients, also call it over plain JSON-RPC (uses curl):

COOKIE=$(cat $DATA_DIR_32/regtest/.cookie)
curl -s --user "$COOKIE" -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"rpc.discover","params":[]}' \
  http://127.0.0.1:18443/ | jq .result.info

Output:

{
  "title": "Bitcoin Core JSON-RPC",
  "version": "v32.0.0rc2",
  "description": "Autogenerated from Bitcoin Core RPC metadata."
}

With show_hidden=true, hidden commands such as echo and setmocktime should be added. Compare the method counts:

bcli32 getopenrpcinfo | jq '.methods | length'
bcli32 -named getopenrpcinfo show_hidden=true | jq '.methods | length'

Expected output:

156
175

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

Mempool RPCs (getrawmempool, getmempoolentry, testmempoolaccept, submitpackage) now include an additional field vsize_adjusted (which is the sigop-adjusted virtual size used for policy) and vsize_bip141 (which represents the raw BIP141 virtual size).

getrawtransaction RPC now includes an additional field vsize_adjusted, which is the sigop-adjusted virtual size if the transaction is in the mempool. #32800

Start Bitcoin Core 32.0 in regtest, generate a wallet, address, and mine 101 blocks:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon
sleep 5
bcli32 createwallet "satoshi"
bcli32 generatetoaddress 101 $(bcli32 getnewaddress)

Send a transaction to generate a mempool entry:

TXID=$(bcli32 -named sendtoaddress address=$(bcli32 getnewaddress) amount=1 fee_rate=1)
echo $TXID

Now call getmempoolentry to see the updated fields (filtered with jq):

bcli32 getmempoolentry $TXID | jq '{vsize, vsize_adjusted, vsize_bip141}'

The output should be something like:

{
  "vsize": 141,
  "vsize_adjusted": 141,
  "vsize_bip141": 141
}

(Cross)check via getrawtransaction and confirm the field is present and consistent with the mempool entry:

bcli32 getrawtransaction $TXID true | jq '{vsize, vsize_adjusted}'

The output should equal to the output of getmempoolentry without the vsize_bip141 entry:

{
  "vsize": 141,
  "vsize_adjusted": 141
}

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

createpsbt, walletcreatefundedpsbt, converttopsbt, and psbtbumpfee will now default to creating version 2 PSBTs. An optional psbt_version argument is added to these RPCs which allows specifying the version of PSBT to create. #21283

Start Bitcoin Core 32.0 in regtest, generate a wallet, address, and mine 101 blocks:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon
sleep 5
bcli32 createwallet "satoshi"
bcli32 generatetoaddress 101 $(bcli32 getnewaddress)

Create a funded PSBT from the 'satoshi' wallet:

FUNDED_PSBT=$(bcli32 walletcreatefundedpsbt "[]" "[{\"$(bcli32 getnewaddress)\":1}]" 0 "{\"fee_rate\":1}" | jq -r '.psbt')
bcli32 decodepsbt $FUNDED_PSBT | jq '.psbt_version'

The expected output is 2, this confirms that the wallet itself now creates v2 PSBTs by default.

2

Now create a v1 PSBT explicitly by using -named to set the psbt_version:

PSBT_V1=$(bcli32 -named createpsbt inputs="[]" outputs="[]" psbt_version=0)
bcli32 decodepsbt $PSBT_V1 | jq '.psbt_version'

The output is somewhat counterintuitive namely 0 instead of 1. This is the correct output (PSBT v0/v1 is internally versioned as 0):

0

To make sure the complete fund -> sign -> finalize -> broadcast pipeline still works with v2, sign the PBST:

SIGNED_PSBT=$(bcli32 -rpcwallet=satoshi walletprocesspsbt $FUNDED_PSBT | jq -r '.psbt')
bcli32 decodepsbt $SIGNED_PSBT | jq '.inputs[0].partial_signatures // .inputs[0].final_scriptwitness'

Expected: signature data present, confirming signing still works on the v2 format.

And Finalize and broadcast the PBST:

FINAL_HEX=$(bcli32 finalizepsbt $SIGNED_PSBT | jq -r '.hex')
bcli32 sendrawtransaction $FINAL_HEX

The output should be a transaction ID (txid), below is an example of a transaction ID (your output should not be equal):

89be67d3cdf04192c718e6f1e6486b02172c53f5f39b0ab73de1c53b0707f1e3

This confirms that the full fund -> sign -> finalize -> broadcast pipeline still works end-to-end with v2 as the default.

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

getbalances now includes a nonmempool field under mine, reporting the net balance of wallet transactions that are not in the mempool (e.g. due to too low a feerate, too many unconfirmed ancestors, or a too-large OP_RETURN output). This value is always negative, and is typically an over-estimate since it accounts for coins being spent but not for change outputs returning to the wallet (which are also outside the mempool). Without this field, funds involved in non-mempool transactions could appear to go missing from the wallet balance. #33671

Start Bitcoin Core 32.0 in regtest, generate a wallet, address, and mine 101 blocks:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon -datacarriersize=0
sleep 5
bcli32 createwallet "satoshi"
bcli32 generatetoaddress 101 $(bcli32 getnewaddress)

Baseline, check that we start with a 0 balance.

bcli32 getbalances | jq '.mine'

the output should be:

{
  "trusted": 50.00000000,
  "untrusted_pending": 0E-8,
  "immature": 5000.00000000,
  "nonmempool": 0E-8
}

Now create a transaction that gets rejected by the mempool:

bcli32 -named send outputs='{"data":"4368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73"}' fee_rate=1

Now validate that the nonmempool transactions is -50:

bcli32 getbalances | jq '.mine'

Output:

{
  "trusted": 50.00000000,
  "untrusted_pending": 0E-8,
  "immature": 5000.00000000,
  "nonmempool": -50.00000000
}

Restart the node with a default datacarriersize so that the transaction will be accepted:

bcli32 stop
sleep 5
bitcoind32 -daemon 

See that the transaction is now accepted and nonmempool is 0 again:

bcli32 loadwallet satoshi
bcli32 getbalances | jq '.mine'

The output should be :

{
  "trusted": 49.99999820,
  "untrusted_pending": 0E-8,
  "immature": 5000.00000000,
  "nonmempool": 0E-8
}

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

The dumptxoutset RPC now supports writing to a named pipe on UNIX-like systems (see mkfifo(1) and mkfifo(3) man pages). This allows the raw UTXO set data to be consumed directly by another process (e.g., the contrib/utxo-tools/utxo_to_sqlite.py conversion script) without first writing it to disk. #31560(https://github.com/bitcoin/bitcoin/pull/31560)

Start Bitcoin Core 32.0 in regtest, generate a wallet, address, and mine 101 blocks:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon
sleep 5
bcli32 createwallet "satoshi"
bcli32 generatetoaddress 101 $(bcli32 getnewaddress)

First, dump the UTXO set to a regular file as a baseline:

bcli32 dumptxoutset $DATA_DIR_32/utxo.dat latest

The output should be something like:

{
  "coins_written": 101,
  "base_hash": "..",
  "base_height": 101,
  "path": "/path/to/DATA_DIR_32/utxo.dat",
  "txoutset_hash": "..",
  "nchaintx": 102
}

Let's now create a named pipe:

mkfifo $DATA_DIR_32/utxo.fifo
ls -l $DATA_DIR_32/utxo.fifo

The output should look like this; the leading p marks it as a pipe:":

prw-r--r-- 1 satoshi  staff  0 Sep 24 17:05 /path/to/DATA_DIR_32/utxo.fifo

A write to a pipe blocks until something reads from the other end, so start a reader in the background first. It copies everything coming through the pipe into a regular file:

cat $DATA_DIR_32/utxo.fifo > $DATA_DIR_32/utxo_from_pipe.dat & READER=$!

Dump the UTXO set into the pipe:

bcli32 dumptxoutset $DATA_DIR_32/utxo.fifo latest | jq '{coins_written, base_height, path}'

The output should match the baseline, except that path now points to the pipe::

{
  "coins_written": 101,
  "base_height": 101,
  "path": "/path/to/DATA_DIR_32/utxo.fifo"
}

Wait for the reader to finish, then compare what came through the pipe with the baseline file:

wait $READER
cmp $DATA_DIR_32/utxo.dat $DATA_DIR_32/utxo_from_pipe.dat && echo "identical"

Expected output:

identical

The pipe itself should still be a pipe:

ls -l $DATA_DIR_32/utxo.fifo

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

The new mempool fee rate estimator produces conservative and economical fee rate estimates from the current contents of the mempool. It only produces a fee rate estimate when recent blocks indicate a healthy mempool, and falls back to the higher of the minimum relay fee rate and the current mempool minimum fee rate when the mempool is too sparse. Its statistics are persisted to fees/mempool_policy_estimator.dat and reloaded on startup. estimatesmartfee returns the lower of the two fee rate estimators' results, so the mempool fee rate estimator can only lower the block policy fee rate estimate. The estimatesmartfee RPC now combines two fee rate estimators: the existing block policy fee rate estimator and a new mempool fee rate estimator. #34075

Start Bitcoin Core 32.0 in regtest, generate a wallet, address, and mine 101 blocks:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon
sleep 5
bcli32 createwallet "satoshi"
bcli32 generatetoaddress 101 $(bcli32 getnewaddress)  > /dev/null

Confirm that the new fees directory is still empty or does not exist:

ls -l $DATA_DIR_32/regtest/fees/

Output:

ls: /path/to/DATA_DIR_32/regtest/fees/: No such file or directory

On a node without a blockhistory there is also no fee history, so no estimate should be produced:

bcli32 estimatesmartfee 2

Output:

{
  "errors": [
    "Insufficient data or no feerate found"
  ],
  "blocks": 0
}

Let's build some fee history, the block policy estimator ignores transactions that spend unconfirmed outputs, so first split the wallet's funds into 100 outputs and then mine a block:

AMOUNTS=$(for i in $(seq 1 100); do echo "\"$(bcli32 getnewaddress)\":0.4"; done | paste -sd, -)
bcli32 -named sendmany amounts="{$AMOUNTS}" fee_rate=1 > /dev/null
bcli32 generatetoaddress 1 $(bcli32 getnewaddress) > /dev/null

Now run 30 rounds of 10 transactions at random fee rates between 1 and 50 sat/vB, mining a block after each round (this takes about a minute):

for round in $(seq 1 30); do
  for i in $(seq 1 10); do
    bcli32 -named sendtoaddress address=$(bcli32 getnewaddress) amount=0.01 fee_rate=$((RANDOM % 50 + 1)) > /dev/null
  done
  bcli32 generatetoaddress 1 $(bcli32 getnewaddress) > /dev/null
done

Now the fee estimator should work, get the estimate from each mode:

NONE=$(bcli32 -named estimatesmartfee conf_target=2 | jq .feerate)
BP=$(bcli32 -named estimatesmartfee conf_target=2 options='{"fee_rate_estimator":"block_policy"}' | jq .feerate)
MP=$(bcli32 -named estimatesmartfee conf_target=2 options='{"fee_rate_estimator":"mempool_policy"}' | jq .feerate)
echo "none=$NONE block_policy=$BP mempool_policy=$MP"

Output:

none=0.00000100 block_policy=0.00004000 mempool_policy=0.00000100

The combined estimate should be exactly the lower of the two:

jq -n "$NONE == ([$BP, $MP] | min)"

Expected output:

true

In the default mode, the response should output which estimator produced the result:

bcli32 estimatesmartfee 2 | jq .estimator

Output:

"mempool_policy"

A verbosity of 2 or higher should contain the mempool health statistics:

bcli32 -named estimatesmartfee conf_target=2 options='{"verbosity":2}' | jq 'has("mempool_health_statistics")'

Expected output:

true

Now that estimates exist, the wallet can spend without an explicit fee_rate. The fee_reason field names why that fee rate was chosen (fee rate estimator, mempool minimum, fallback, or minimum required):

bcli32 -named sendtoaddress address=$(bcli32 getnewaddress) amount=0.01 verbose=true | jq .fee_reason

Stop the node and confirm that both estimator files were written:

bcli32 stop && sleep 5
ls $DATA_DIR_32/regtest/fees/

Expected output:

block_policy_estimates.dat  mempool_policy_estimator.dat

Check that the old fee estimates are migrated correctly, do this by moving the block policy file back to the legacy location:

mv $DATA_DIR_32/regtest/fees/block_policy_estimates.dat $DATA_DIR_32/regtest/fee_estimates.dat
bitcoind32 -daemon && sleep 5
ls $DATA_DIR_32/regtest/fees/ $DATA_DIR_32/regtest/fee_estimates.dat

Expected output, block_policy_estimates.dat should be back in fees/, and fee_estimates.dat should have been removed:

ls: /path/to/$DATA_DIR_32/regtest/fee_estimates.dat: No such file or directory
/path/to/$DATA_DIR_32/regtest/fees/:
block_policy_estimates.dat      mempool_policy_estimator.dat

Confirm the migrated estimates still work:

bcli32 -named estimatesmartfee conf_target=2 options='{"fee_rate_estimator":"block_policy"}' | jq .feerate

Output:

0.00004000

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

We are going to test these new options in tandem;

A new RPC addhdkey is added which allows a BIP 32 extended key to be added to the wallet without needing to import it as part of a separate descriptor. This key will not be used to produce any output scripts unless it is explicitly imported as part of a separate descriptor independent of the addhdkey RPC. #29136

A new derivehdkey RPC is available to obtain an xpub or xprv for a derivation path with at least one hardened step from an HD key known to the wallet. This can be used to coordinate a multisig setup, where each signer shares an xpub using a different derivation path than the default single-signature descriptors. The example in doc/multisig-tutorial.md is updated to use this RPC. #32784

Start Bitcoin Core 32.0 in regtest, generate a wallet:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon
sleep 5
bcli32 createwallet "satoshi"

Validate that there are no unused descriptors in the current wallet (this should give no output):

bcli32 listdescriptors | jq '.descriptors[] | select(.desc|test("unused"))'

Add a new BIP32 extended key by using the new addhdkey:

ADDED_HDKEY=$(bcli32 addhdkey)
echo "$ADDED_HDKEY"

This should give back something like:

{
  "xpub": "tpub..."
}

Validate that there is now an unused and inactive descriptor:

bcli32 listdescriptors | jq '.descriptors[] | select(.desc|test("unused"))'

output:

{
  "desc": "unused(tpub...)#dededede",
  "timestamp": 17...,
  "active": false
}

With addhdkey you can also specify an xprv to add, lets validate that addhdkey does not accept an xpub:

NEW_XPUB=$(echo "$ADDED_HDKEY" | jq -r .xpub)
echo "$NEW_XPUB"
bcli32 addhdkey "$NEW_XPUB"

The output should be:

error code: -5
error message:
Extended public key (xpub) provided, but extended private key (xprv) is required

We have now validated that the new addhdkey RPC works, lets continue with derivehdkey

Default derivehdkey uses the unused key (we just added), you only have to specify the derivation path:

DERIVED=$(bcli32 derivehdkey "m/87h/0h/0h")
echo "$DERIVED"

The output should be something like:

{
  "origin": "[dededede/87h/0h/0h]",
  "xpub": "tpub..."
}

Where origin is your own fingerprint and matching derivation path.

Using the named version with explicitly setting the added key should result in the same outcome:

DERIVED_PRIV=$(bcli32 -named derivehdkey path="m/87h/0h/0h" private=false hdkey="$NEW_XPUB")
echo "$DERIVED_PRIV"

Comparing the diff should give no output (they are equal)

diff <(echo "$DERIVED" | jq .xpub) <(echo "$DERIVED_PRIV" | jq .xpub)

Setting the private to true, will also generate a xprv:

DERIVED_PRIV=$(bcli32 -named derivehdkey path="m/87h/0h/0h" private=true hdkey="$NEW_XPUB")
echo "$DERIVED_PRIV"

The output should now give next to the "origin" and "xpub" also an "xprv":

{
  "origin": "[dededede/87h/0h/0h]",
  "xpub": "tpub...",
  "xprv": "tprv..."
}

Calling the derivehdkey with a derivation path that has no hardened step, should result in an error:

bcli32 derivehdkey "m/0/0"

Output:

error code: -8
error message:
Derivation path requires at least one hardened step

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

HTTP server has been completely rewritten to replace/remove libevent from the code. In this effort some behavior had to be changed to better adhere to the RFC specs and improve security. Although a lot of testing has been done to test client compatibility please test your client against these behavioral changes.

The libevent logging category has been removed. Configurations like -debug=libevent or -debugexclude=libevent will log a deprecation warning and be ignored. These configurations will result in an error in a future release.

A new configuration option -rpcmaxconnections (default 16) limits the number of simultaneously connected HTTP clients to the server. The application will now attempt to reserve file descriptors for the HTTP server sockets. If your system has limited resources, consider using a lower setting. #35182

Start Bitcoin Core 32.0 in regtest with rest enabled, generate a wallet, mine a block:

echo "regtest=1" > $DATA_DIR_32/bitcoin.conf
bitcoind32 -daemon -rest
sleep 5
bcli32 createwallet "satoshi"
bcli32 generatetoaddress 1 $(bcli32 getnewaddress)

Set up the RPC/REST address (regtest defaults to port 18443) and the cookie for authenticated calls:

URL="http://127.0.0.1:18443"
COOKIE=$(cat $DATA_DIR_32/regtest/.cookie)

Confirm that this does not impact RPC:

bcli32 getblockchaininfo | jq '{chain, blocks}'

Should return:

{                                                                                                                           
 "chain": "regtest",
 "blocks": 1
}

Confirm that normal JSON-RPC over HTTP works:

curl -s --user "$COOKIE" -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"getblockchaininfo","params":[]}' $URL/ | jq '.result | {chain,blocks}'

Should return the same output:

{                                                                                                                           
 "chain": "regtest",
 "blocks": 1
}

Confirm that a well-formed raw request is accepted by the node:

 (printf 'GET /rest/chaininfo.json HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n'; sleep 1) | nc 127.0.0.1 18443 | head -1

Output:

HTTP/1.1 200 OK

A Header size just under the limit (8192-byte limit) should be accepted (HTTP 200):

curl -s -o /dev/null -w "%{http_code}\n" -H "X-Big: $(head -c 8000 /dev/zero | tr '\0' 'a')" $URL/rest/chaininfo.json

Output:

200

A Header size over the limit (8192-byte limit) should be rejected (HTTP 400):

curl -s -o /dev/null -w "%{http_code}\n" -H "X-Big: $(head -c 9000 /dev/zero | tr '\0' 'a')" $URL/rest/chaininfo.json

Output:

400

Whitespace in a header field-name should be rejected (HTTP 400):

(printf 'GET /rest/chaininfo.json HTTP/1.1\r\nHost: localhost\r\nBad Header: x\r\nConnection: close\r\n\r\n'; sleep 1) | nc 127.0.0.1 18443 | head -1

Output:

HTTP/1.1 400 Bad Request

Line folding (a header line starting with whitespace), which should be rejected:

(printf 'GET /rest/chaininfo.json HTTP/1.1\r\nHost: localhost\r\nX-Test: a\r\n continued\r\nConnection: close\r\n\r\n'; sleep 1) | nc 127.0.0.1 18443 | head -1

Output:

HTTP/1.1 400 Bad Request

Two Content-Length headers with different values also should be rejected:

(printf 'POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length: 5\r\nContent-Length: 6\r\nConnection: close\r\n\r\nhello'; sleep 1) | nc 127.0.0.1 18443 | head -1

Output:

HTTP/1.1 400 Bad Request

A trailing % in the URL, which should now be tolerated. You should get a normal REST error (for example 404) rather than a 400:

curl -s -o /dev/null -w "%{http_code}\n" "$URL/rest/chaininfo.json%"

Output:

404

Restart the node with a low connection limit:

bcli32 stop && sleep 5
bitcoind32 -daemon -rest -rpcmaxconnections=2

Hold two idle connections open in the background and try to connect a third client. That client should now be refused or blocked until a slot frees up:

(printf 'GET /rest/chaininfo.json HTTP/1.1\r\nHost: localhost\r\n'; sleep 20) | nc 127.0.0.1 18443 > /dev/null & NC1=$!
(printf 'GET /rest/chaininfo.json HTTP/1.1\r\nHost: localhost\r\n'; sleep 20) | nc 127.0.0.1 18443 > /dev/null & NC2=$!
sleep 1
echo "established: $(lsof -nP -iTCP:18443 -sTCP:ESTABLISHED | grep -c '^nc')"
echo "third client: $(curl -s -m 5 -o /dev/null -w '%{http_code}' $URL/rest/chaininfo.json)"
kill $NC1 $NC2
sleep 1
echo "after release: $(curl -s -o /dev/null -w '%{http_code}' $URL/rest/chaininfo.json)"

Expected output:

established: 2
third client: 000
after release: 200

Advanced test:

Localhost is always allowed, so this test needs a non-loopback address on your machine, such as your LAN IP. Replace with the lan ip and execute:

LAN_IP=<your LAN IP>

Start a node that binds RPC to the LAN IP and allow only an unrelated IPs:

bcli32 stop && sleep 5
bitcoind32 -daemon -rest -rpcbind=127.0.0.1 -rpcbind=$LAN_IP -rpcallowip=192.0.2.1
sleep 5
curl -s -o /dev/null -w "%{http_code}\n" http://$LAN_IP:18443/rest/chaininfo.json; echo "curl exit: $?"

Expected output:

000
curl exit: 56

The connection should be closed immediately with no HTTP response. curl should report Connection reset by peer (error 56). You should not see 403 Forbidden, which was the old behavior.

Stop the node and clean up:

bcli32 stop && sleep 5  &&  datadir-cleanup

TBD

Kudos if you made it this far 👏🎉

Thanks for your contribution and for taking the time to make this Bitcoin release awesome.

For feedback on this guide, please visit this issue.