December 2025

2025-12

Sebastian made some attempt to remove one of the volatile (i.e. non-persisted) input queues in a potential ADR and investigation; but decided not to proceed for now.

2025-12-01

SB on partial fanout

Continuing to debug to figure out what was the problem with decommit/close combination.
I'd like to print the utxo right before we cover the fee in the wallet and see why does the output change the expected value.
What I find weird is - we call a ledger function ensureMinCoinTxOut and that one changes the head output to contain slightly more ada (around 34480 lovelace more) but in case the decommit was not done partially this function keeps the output the same! Why? I don't see any additional context a part from protocol parameters that could affect the output value.
One thing I notice is that the output size in case of a normal decommit test run is smaller than the one where we decommit partially. It might be because our snapshot contains two records for utxoToDecommit and this affects the close validator where we check the input/output values?
But if datum contents affect the output balancing how come we didn't run into this problem before?
I mean we do have one more extra field for the accumulator so perhaps this in combination with the snapshot being a bit bigger since there are two records in utxoToDecommit might cause re-balancing to happen.
In this case it would be perfectly fine to request from the validators that outputs need to be >= instead of ==
There are problems with this because Value doesn't have Ord instance so we could only compare lovelace value on-chain.
Perhaps it is doable to check if lovelace amount is >= and make sure any extra tokens are the same between input/output?
This would grow the script even more so we need to be aware of that.
I will not spend any more time on this since I will not get to the bottom of this - my proof for this theory can be ensureMinCoinTxOut is altering the output thus it means the output is too small. Why is it too small? Well we had a change in our close datum which grows the datum bigger and when the snapshot contains two records for utxoToCommit the datum is also a bit bigger so this combination is making the close fail. If there is only one record in the utxoToDecommit all works out fine so it seem we are balancing on the edge here when it comes to output lovelace.

November 2025

2025-11-26

SB on debugging close transaction

Our close tx fails because of unknown error (after cardano network problems on preview):

{"timestamp":"2025-11-26T08:57:38.332743706Z","threadId":545143,"namespace":"HydraNode-\"Sasha\"","message":{"api":{"sentOutput":{"postChainTx":{"closingSnapshot":{"signatures":{"multiSignature":["572d79dd34fabfeeafe8c4b76e54c669f0408ecd021cd37538cefeb7a35ba19c9c033586ed70107785b2606e1505ae010da229481e662220b83cd42dbba57206","d810768ed709562ac2fe1c777410fd4f2f370936ea5854de511c5b7bbac1131f70f650e0563b36aaa8f8bf34307e4d0e54cefea8e0176b1408f1a0b6b62f8e02","9756636d6ef342a7f8d5add94a5aa955c88fb6605421c4431b133a70b99d92f3a492a41b3df2495bc243893299e023f57ae81e8a44e9011145c442108c2fa203","98b0d827083f2463bee3588effd096f134faa21bedb188a11f782c2cafb15d957b3abf02e6d139abe30ad31b90d30e26f4e92b683f11d4bd1b9328a928bb9e0f","145281583d3bfea566cac3d62b8ac1d9b20c553f639e7049f70db629d69e85415dea2ee8a48931aac5ccc640dded3a539f6bf9ca64d3581b430864c4636e480d","e2ed4f6a4ee10853c59963991c5ad554a9254d9ca1a87d1eb260f192796eb05a36504616d1df4aac0adccfe463259d5120ff7ce58afd4c080c15190cb1626201"]},"snapshot":{"confirmed":[{"cborHex":"84a300d9010281825820dd0a12aaf853cb9520a71af8052dd25de39a0602b2e333412507cfdc4a05b56d01018282581d600f93fbedf31223a2c7529b63cd46038db1e07338cc59f075f8a7b1ef1a3b9aca0082581d600f93fbedf31223a2c7529b63cd46038db1e07338cc59f075f8a7b1ef1a3b9aca000200a100d9010281825820584896516d4030680ebfaf29e29c21eb5a6e2ec3b007a0b5b948fc47782746af584006596eec2a0d6030eee1bd9b653293d044616235993707a2c8b47212b69ac7f89babc80884b9421464d3175f7f6721dfc9efa5716751a97dbc6ae0ae65449408f5f6","description":"","txId":"23f39bd04c07958e39e78012bab410ddda0a95ea8770d3245b8f19e6f2187c28","type":"Tx ConwayEra"}],"headId":"2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a","number":4,"utxo":{"23f39bd04c07958e39e78012bab410ddda0a95ea8770d3245b8f19e6f2187c28#0":{"address":"addr_test1vq8e87ld7vfz8gk822dk8n2xqwxmrcrn8rx9nur4lznmrmcldczr3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000000}},"23f39bd04c07958e39e78012bab410ddda0a95ea8770d3245b8f19e6f2187c28#1":{"address":"addr_test1vq8e87ld7vfz8gk822dk8n2xqwxmrcrn8rx9nur4lznmrmcldczr3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000000}},"6db2cfb6c2fcc6b62cf5d5c371f36d1c9834f8db2681961b906881b144d8d7e1#0":{"address":"addr_test1vqm4sxc8fsfkycu0kjhyl4m8uaj9zy428taegs3wh9jl62c97ztr7","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":2000000}},"d42607a6c6190e79546794ec7ba37419161cc2d9ea541cd17ba406ea0c413072#0":{"address":"addr_test1vpkdvdpl2zrh05x6z9d6dept0350s02p5tch6aet97zfgvsjgutah","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":100}},"d42607a6c6190e79546794ec7ba37419161cc2d9ea541cd17ba406ea0c413072#1":{"address":"addr_test1vqm4sxc8fsfkycu0kjhyl4m8uaj9zy428taegs3wh9jl62c97ztr7","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9999999900}},"dd0a12aaf853cb9520a71af8052dd25de39a0602b2e333412507cfdc4a05b56d#0":{"address":"addr_test1vpkdvdpl2zrh05x6z9d6dept0350s02p5tch6aet97zfgvsjgutah","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000000}}},"utxoToCommit":null,"utxoToDecommit":null,"version":0},"tag":"ConfirmedSnapshot"},"headId":"2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a","headParameters":{"contestationPeriod":300,"parties":[{"vkey":"a65dd1af5d938e394b05198b97870a33343d3dd2ec8feb8906a2cbbd2cc0ac96"},{"vkey":"32bce7668b68bcc66750963217e50244fb7a9495053fb787fd5480aa1e509366"},{"vkey":"935df47d312772df391cc7a934702e13e9f4263c611cc275b93f76ec6d31d02c"},{"vkey":"9e0e0a73d7e878d6719d374c667e99ff2b5e933130d4ca67a335cca06443a34d"},{"vkey":"8c69cf05a794ae72151dbd7522d9398daf30b14d0a8f9972e31f76fb14b0b4f1"},{"vkey":"6865953562830aee1faaedf2353aed54519743d9c427f2f75a89a8485e3d60ee"}]},"openVersion":0,"tag":"CloseTx"},"postTxError":{"failingTx":{"cborHex":"84aa00d9010282825820240be58f60c313a192e1c6d252d13e48f75c786092ce728601cb36c549198d6c0082582038e24bb01bd5f1275699a3ede314fd4541a2e58cf69ac766c8793fc3ac900c33010dd901028182582038e24bb01bd5f1275699a3ede314fd4541a2e58cf69ac766c8793fc3ac900c330112d90102818258200b32f7cf144090b3a2d6787cb5b4cabbc0a72c1ae77bf72de8e3d9aa9476bfb7000182a300581d70a1442faf26d4ec409e2f62a685c1d4893f8d6bcbaf7bcb59d6fa134001821b00000003079c6d3ca1581c2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636aa74b487964726148656164563101581c4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b4101581c5ecf89bf0e00eaf8667d9ff9bb92c6faac2fbcc6ff71ce9a8d6c503301581c6cd6343f508777d0da115ba6e42b7c68f83d41a2f17d772b2f84943201581c96c6d56eb5b0b80595fa11df3cccbdf8523a0f6d869ea162be0fde2301581cda0efb26cb9b09c64a3725704609d3ede6d72dc985d10e10dc6a19c701581cdb71b965a22cf2849a903f81c5aba1ccd9108f2b636477351c5b13c201028201d81859016fd87b9fd8799f581c2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a9f5820a65dd1af5d938e394b05198b97870a33343d3dd2ec8feb8906a2cbbd2cc0ac96582032bce7668b68bcc66750963217e50244fb7a9495053fb787fd5480aa1e5093665820935df47d312772df391cc7a934702e13e9f4263c611cc275b93f76ec6d31d02c58209e0e0a73d7e878d6719d374c667e99ff2b5e933130d4ca67a335cca06443a34d58208c69cf05a794ae72151dbd7522d9398daf30b14d0a8f9972e31f76fb14b0b4f158206865953562830aee1faaedf2353aed54519743d9c427f2f75a89a8485e3d60eeffd8799f1a000493e0ff000458206a79eae4dcf0d3233b1d58e84f22e4db4884d82bafab1ec51d086ce6d4f739135820e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b8555820e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855801b0000019abf63a168ffff82581d604fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b411b00000003d0faba50021a00206870031a05cf9945081a05cf987d0ed9010281581c4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b410b58203578239c2cb861d6b5e3f48f7d8e57989df07eca82241ec34c25086cb9ccd809075820f5cf4df0aba78e42562e6b5ba81d6663a8e039210a0f3c3c711cfb6cbeef6e51a200d9010281825820f4e4d4d2c478420c70b57d38c02514d73537bb58ec651885df45f95c1ea429fe5840b8066174e46f4fd1c2d80baffa3b1c18088a8f45a0e267600cba4ce277837689dc88507a408eb59c7a2ecfa2283d2d3e17d3396156b290bd2f139e675186f00b05a182000082d87c9fd87a9f9f5840572d79dd34fabfeeafe8c4b76e54c669f0408ecd021cd37538cefeb7a35ba19c9c033586ed70107785b2606e1505ae010da229481e662220b83cd42dbba572065840d810768ed709562ac2fe1c777410fd4f2f370936ea5854de511c5b7bbac1131f70f650e0563b36aaa8f8bf34307e4d0e54cefea8e0176b1408f1a0b6b62f8e0258409756636d6ef342a7f8d5add94a5aa955c88fb6605421c4431b133a70b99d92f3a492a41b3df2495bc243893299e023f57ae81e8a44e9011145c442108c2fa203584098b0d827083f2463bee3588effd096f134faa21bedb188a11f782c2cafb15d957b3abf02e6d139abe30ad31b90d30e26f4e92b683f11d4bd1b9328a928bb9e0f5840145281583d3bfea566cac3d62b8ac1d9b20c553f639e7049f70db629d69e85415dea2ee8a48931aac5ccc640dded3a539f6bf9ca64d3581b430864c4636e480d5840e2ed4f6a4ee10853c59963991c5ad554a9254d9ca1a87d1eb260f192796eb05a36504616d1df4aac0adccfe463259d5120ff7ce58afd4c080c15190cb1626201ffffff821a00fbc5201b00000002540be400f5d90103a100a119d9036f487964726156312f436c6f73655478","description":"","txId":"26cf6ab421d6c6fa3bb051c482c756375a95b6fe472bbcb2171dacc8c7818629","type":"Tx ConwayEra"},"failureReason":"TxValidationErrorInCardanoMode (ShelleyTxValidationError ShelleyBasedEraConway (ApplyTxError (ConwayUtxowFailure (UtxoFailure (InsufficientCollateral (DeltaCoin 0) (Coin 3185832))) :| [ConwayUtxowFailure (UtxoFailure NoCollateralInputs),ConwayUtxowFailure (UtxoFailure (BadInputsUTxO (fromList [TxIn (TxId {unTxId = SafeHash \"240be58f60c313a192e1c6d252d13e48f75c786092ce728601cb36c549198d6c\"}) (TxIx {unTxIx = 0}),TxIn (TxId {unTxId = SafeHash \"38e24bb01bd5f1275699a3ede314fd4541a2e58cf69ac766c8793fc3ac900c33\"}) (TxIx {unTxIx = 1})]))),ConwayUtxowFailure (UtxoFailure (ValueNotConservedUTxO (Mismatch {mismatchSupplied = MaryValue (Coin 0) (MultiAsset (fromList [])), mismatchExpected = MaryValue (Coin 29405712380) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash \"2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a\"},fromList [(\"4879647261486561645631\",1),(\"4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b41\",1),(\"5ecf89bf0e00eaf8667d9ff9bb92c6faac2fbcc6ff71ce9a8d6c5033\",1),(\"6cd6343f508777d0da115ba6e42b7c68f83d41a2f17d772b2f849432\",1),(\"96c6d56eb5b0b80595fa11df3cccbdf8523a0f6d869ea162be0fde23\",1),(\"da0efb26cb9b09c64a3725704609d3ede6d72dc985d10e10dc6a19c7\",1),(\"db71b965a22cf2849a903f81c5aba1ccd9108f2b636477351c5b13c2\",1)])]))}))),ConwayUtxowFailure (PPViewHashesDontMatch (Mismatch {mismatchSupplied = SJust (SafeHash \"3578239c2cb861d6b5e3f48f7d8e57989df07eca82241ec34c25086cb9ccd809\"), mismatchExpected = SJust (SafeHash \"884a124ee030dd4b55c04015c2ca2e213f12590b9dd42e983f82b1b7558eb1e2\")})),ConwayUtxowFailure (ExtraRedeemers [ConwaySpending (AsIx {unAsIx = 0})])])))","tag":"FailedToPostTx"},"tag":"PostTxOnChainFailed"},"tag":"APIOutputSent"},"tag":"APIServer"}}

We want to understand why this happens even if the problem is not in Hydra since it is a useful exercise.
This is how our close tx looks like:

  
 {
    "auxiliary scripts": null,
    "certificates": null,
    "collateral inputs": [
        "38e24bb01bd5f1275699a3ede314fd4541a2e58cf69ac766c8793fc3ac900c33#1"
    ],
    "currentTreasuryValue": null,
    "datums": [],
    "era": "Conway",
    "fee": "2123888 Lovelace",
    "governance actions": [],
    "inputs": [
        "240be58f60c313a192e1c6d252d13e48f75c786092ce728601cb36c549198d6c#0",
        "38e24bb01bd5f1275699a3ede314fd4541a2e58cf69ac766c8793fc3ac900c33#1"
    ],
    "metadata": {
        "55555": "HydraV1/CloseTx"
    },
    "mint": null,
    "outputs": [
        {
            "address": "addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3",
            "address era": "Shelley",
            "amount": {
                "lovelace": 13012593980,
                "policy 2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a": {
                    "asset 4879647261486561645631 (HydraHeadV1)": 1,
                    "asset 4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b41": 1,
                    "asset 5ecf89bf0e00eaf8667d9ff9bb92c6faac2fbcc6ff71ce9a8d6c5033": 1,
                    "asset 6cd6343f508777d0da115ba6e42b7c68f83d41a2f17d772b2f849432": 1,
                    "asset 96c6d56eb5b0b80595fa11df3cccbdf8523a0f6d869ea162be0fde23": 1,
                    "asset da0efb26cb9b09c64a3725704609d3ede6d72dc985d10e10dc6a19c7": 1,
                    "asset db71b965a22cf2849a903f81c5aba1ccd9108f2b636477351c5b13c2": 1
                }
            },
            "datum": {
                "constructor": 2,
                "fields": [
                    {
                        "constructor": 0,
                        "fields": [
                            {
                                "bytes": "2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a"
                            },
                            {
                                "list": [
                                    {
                                        "bytes": "a65dd1af5d938e394b05198b97870a33343d3dd2ec8feb8906a2cbbd2cc0ac96"
                                    },
                                    {
                                        "bytes": "32bce7668b68bcc66750963217e50244fb7a9495053fb787fd5480aa1e509366"
                                    },
                                    {
                                        "bytes": "935df47d312772df391cc7a934702e13e9f4263c611cc275b93f76ec6d31d02c"
                                    },
                                    {
                                        "bytes": "9e0e0a73d7e878d6719d374c667e99ff2b5e933130d4ca67a335cca06443a34d"
                                    },
                                    {
                                        "bytes": "8c69cf05a794ae72151dbd7522d9398daf30b14d0a8f9972e31f76fb14b0b4f1"
                                    },
                                    {
                                        "bytes": "6865953562830aee1faaedf2353aed54519743d9c427f2f75a89a8485e3d60ee"
                                    }
                                ]
                            },
                            {
                                "constructor": 0,
                                "fields": [
                                    {
                                        "int": 300000
                                    }
                                ]
                            },
                            {
                                "int": 0
                            },
                            {
                                "int": 4
                            },
                            {
                                "bytes": "6a79eae4dcf0d3233b1d58e84f22e4db4884d82bafab1ec51d086ce6d4f73913"
                            },
                            {
                                "bytes": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
                            },
                            {
                                "bytes": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
                            },
                            {
                                "list": []
                            },
                            {
                                "int": 1764147569000
                            }
                        ]
                    }
                ]
            },
            "network": "Testnet",
            "payment credential script hash": "a1442faf26d4ec409e2f62a685c1d4893f8d6bcbaf7bcb59d6fa1340",
            "reference script": null,
            "stake reference": null
        },
        {
            "address": "addr_test1vp8m20m650s8u0em4pgka569zx0fssvzywzuqvrnzysfksgxeq2s2",
            "address era": "Shelley",
            "amount": {
                "lovelace": 16390994512
            },
            "network": "Testnet",
            "payment credential key hash": "4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b41",
            "reference script": null,
            "stake reference": null
        }
    ],
    "redeemers": [
        {
            "purpose": {
                "spending script witnessed input": "240be58f60c313a192e1c6d252d13e48f75c786092ce728601cb36c549198d6c#0"
            },
            "redeemer": {
                "data": "Constr 3 [Constr 1 [List [B \"W-y\\221\\&4\\250\\191\\238\\175\\232\\196\\183nT\\198i\\240@\\142\\205\\STX\\FS\\211u8\\206\\254\\183\\163[\\161\\156\\156\\ETX5\\134\\237p\\DLEw\\133\\178`n\\NAK\\ENQ\\174\\SOH\\r\\162)H\\RSf\\\" \\184<\\212-\\187\\165r\\ACK\",B \"\\216\\DLEv\\142\\215\\tV*\\194\\254\\FSwt\\DLE\\253O/7\\t6\\234XT\\222Q\\FS[{\\186\\193\\DC3\\USp\\246P\\224V;6\\170\\168\\248\\191\\&40~M\\SOT\\206\\254\\168\\224\\ETBk\\DC4\\b\\241\\160\\182\\182/\\142\\STX\",B \"\\151Vcmn\\243B\\167\\248\\213\\173\\217JZ\\169U\\200\\143\\182`T!\\196C\\ESC\\DC3:p\\185\\157\\146\\243\\164\\146\\164\\ESC=\\242I[\\194C\\137\\&2\\153\\224#\\245z\\232\\RS\\138D\\233\\SOH\\DC1E\\196B\\DLE\\140/\\162\\ETX\",B \"\\152\\176\\216'\\b?$c\\190\\227X\\142\\255\\208\\150\\241\\&4\\250\\162\\ESC\\237\\177\\136\\161\\USx,,\\175\\177]\\149{:\\191\\STX\\230\\209\\&9\\171\\227\\n\\211\\ESC\\144\\211\\SO&\\244\\233+h?\\DC1\\212\\189\\ESC\\147(\\169(\\187\\158\\SI\",B \"\\DC4R\\129X=;\\254\\165f\\202\\195\\214+\\138\\193\\217\\178\\fU?c\\158pI\\247\\r\\182)\\214\\158\\133A]\\234.\\232\\164\\137\\&1\\170\\197\\204\\198@\\221\\237:S\\159k\\249\\202d\\211X\\ESCC\\bd\\196cnH\\r\",B \"\\226\\237OjN\\225\\bS\\197\\153c\\153\\FSZ\\213T\\169%M\\156\\161\\168}\\RS\\178`\\241\\146yn\\176Z6PF\\SYN\\209\\223J\\172\\n\\220\\207\\228c%\\157Q\\255|\\229\\138\\253L\\b\\f\\NAK\\EM\\f\\177bb\\SOH\"]]]",
                "execution units": {
                    "memory": 16500000,
                    "steps": 10000000000
                }
            }
        }
    ],
    "reference inputs": [
        "0b32f7cf144090b3a2d6787cb5b4cabbc0a72c1ae77bf72de8e3d9aa9476bfb7#0"
    ],
    "required signers (payment key hashes needed for scripts)": [
        "4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b41"
    ],
    "return collateral": null,
    "scripts": [],
    "total collateral": null,
    "treasuryDonation": 0,
    "update proposal": null,
    "validity range": {
        "lower bound": 97491069,
        "upper bound": 97491269
    },
    "voters": {},
    "withdrawals": null,
    "witnesses": [
        {
            "key": "VKey (VerKeyEd25519DSIGN \"f4e4d4d2c478420c70b57d38c02514d73537bb58ec651885df45f95c1ea429fe\")",
            "signature": "SignedDSIGN (SigEd25519DSIGN \"b8066174e46f4fd1c2d80baffa3b1c18088a8f45a0e267600cba4ce277837689dc88507a408eb59c7a2ecfa2283d2d3e17d3396156b290bd2f139e675186f00b\")"
        }
    ]
}

The first input close tx is trying to spend is this one (CollectCom output) https://preview.cexplorer.io/tx/240be58f60c313a192e1c6d252d13e48f75c786092ce728601cb36c549198d6c?tab=content which indeed contains around 13k ada. We are trying to spend output with index 0 here which the Head output.
The second input close tx is trying to spend is this one (Commit output) https://preview.cexplorer.io/tx/38e24bb01bd5f1275699a3ede314fd4541a2e58cf69ac766c8793fc3ac900c33?tab=content , output with index 1! Why? This is commit output that should have been collected into a Head! Explorer even says that this output was already consumed by our CollectCom so why does the hydra-node try to spend it again!
Looking at our close tx construction - it seems like the head UTxO which we try to spend here somehow got the commit utxo still inside. This is our spendableUTxO..huge:

{"05b513d8ab1e4831c726639900da7e4bc9ebb27bf5a9ba0631a8a42381c1e4be#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763167807180},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"34369d6d51a83ac1294348b12f7843e755b75a7486aef6577d3d35c3b99f7eb1"},{"int":1}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa340a1401a006d17e5581c9e9d0ceee3e53813e29585422e0b59a668bb3d0f15528299dc15e66ba158208d278550f81d83d82d0a1d6485ba5cc430d54ce217d69f9030b7a197a00939f201581cfe657d40400fffb4456dc426db0bd49ce755b89da17224c84bd76557a25820c2c0f9a15c1c32291d665be6c68fc08c70d2bfcf7b4c75114c470d9484c4d66a015820f628e4cd6d28348780689f821f7be9241308ecf8c5bf559cdbf5cba570ac380401d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a84fda6cc9fd8799fd8799f582034369d6d51a83ac1294348b12f7843e755b75a7486aef6577d3d35c3b99f7eb101ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58403c8487efbe7c6436a2082203c8d7f5ffffffffa340a1401a006d17e5581c9e9d0ceee3e53813e29585422e0b59a668bb3d0f15528299dc15e66ba158208d2785584050f81d83d82d0a1d6485ba5cc430d54ce217d69f9030b7a197a00939f201581cfe657d40400fffb4456dc426db0bd49ce755b89da17224c84bd76557a25820c25840c0f9a15c1c32291d665be6c68fc08c70d2bfcf7b4c75114c470d9484c4d66a015820f628e4cd6d28348780689f821f7be9241308ecf8c5bf559cdbf5cba570ac4a380401d87980d87a80ffffffffff","inlineDatumhash":"c554e5847b25eaea1ac485784a7248465d2a0e0c42769854253735a5a415d676","referenceScript":null,"value":{"9e9d0ceee3e53813e29585422e0b59a668bb3d0f15528299dc15e66b":{"8d278550f81d83d82d0a1d6485ba5cc430d54ce217d69f9030b7a197a00939f2":1},"fe657d40400fffb4456dc426db0bd49ce755b89da17224c84bd76557":{"c2c0f9a15c1c32291d665be6c68fc08c70d2bfcf7b4c75114c470d9484c4d66a":1,"f628e4cd6d28348780689f821f7be9241308ecf8c5bf559cdbf5cba570ac3804":1},"lovelace":7149541}},"065701b6d7690c27b6a0d8021e941ff035452908802a5ed6abfd5c06caac1b83#0":{"address":"addr_test1vraqpte9yvyufr88vsqdakwdgypyc3qtdxgtnw5j5g8h9cq98ud90","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"065701b6d7690c27b6a0d8021e941ff035452908802a5ed6abfd5c06caac1b83#1":{"address":"addr_test1vraqpte9yvyufr88vsqdakwdgypyc3qtdxgtnw5j5g8h9cq98ud90","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"065701b6d7690c27b6a0d8021e941ff035452908802a5ed6abfd5c06caac1b83#2":{"address":"addr_test1vpp04sgevgwy0ak5uem5d4djam9akyh20kcruc0dfprshdcq46vmj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":29000000}},"065701b6d7690c27b6a0d8021e941ff035452908802a5ed6abfd5c06caac1b83#3":{"address":"addr_test1vpez6ejn2lddp502xer9mdvydffa6ekxew8ttzn73xse0yc486vqr","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19450839}},"08ea69c08a7154b1cc08aeb74b132205660e9c5f63c18821f076be4ee044c44d#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44603401}},"08ea69c08a7154b1cc08aeb74b132205660e9c5f63c18821f076be4ee044c44d#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"0d23708d46a3592b47ea1cc1ff0c805cb9fc98cbd016b346ab366152ea063a2e#1":{"address":"addr_test1vp4mmgwmm8pv9nems5nylnvrmdk6mrcgm0n87ghsntst83gzrcs82","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"11a9f95a8d26b5b6dc06f3230def3ab6f3e71858a924296ef14252ba7008ff3f#1":{"address":"addr_test1vqf2tkxj4jz4gjjjkhvye5q7nmnl9wptvcjc5mzuv0hc90gn8jkf8","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"14fcd77e42ac6803b226ae18c243c3f25b7c30932eb6af727a4915c53c55f697#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":49148192}},"14fcd77e42ac6803b226ae18c243c3f25b7c30932eb6af727a4915c53c55f697#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"1864f936c9ec0a98d7238afa4bca92e73ce0d6724feb7690cd9ff345a79eb3a9#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":10129366}},"1864f936c9ec0a98d7238afa4bca92e73ce0d6724feb7690cd9ff345a79eb3a9#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"19e3f4482f5fe5a54cbf8e92ae5ddeb847d24d65bcdb11446e39b01b100200ea#0":{"address":"addr_test1vpsjcjtkxh0wxpvysx80dn3wr3zhraxr3rzzwrx7q2znkfc2p6fnk","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"19e3f4482f5fe5a54cbf8e92ae5ddeb847d24d65bcdb11446e39b01b100200ea#1":{"address":"addr_test1vpsjcjtkxh0wxpvysx80dn3wr3zhraxr3rzzwrx7q2znkfc2p6fnk","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"19e3f4482f5fe5a54cbf8e92ae5ddeb847d24d65bcdb11446e39b01b100200ea#2":{"address":"addr_test1vzce43eswfck39znvyf6g49ep3hz6z0kwk77g5dm28nkzxgu7yjme","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28000000}},"19e3f4482f5fe5a54cbf8e92ae5ddeb847d24d65bcdb11446e39b01b100200ea#3":{"address":"addr_test1vpsjcjtkxh0wxpvysx80dn3wr3zhraxr3rzzwrx7q2znkfc2p6fnk","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"19e3f4482f5fe5a54cbf8e92ae5ddeb847d24d65bcdb11446e39b01b100200ea#4":{"address":"addr_test1vpxgjycnjprkmurx8yzykft045e4g2l5tngf94jcp9uc5usfcujww","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19449211}},"1ef338b0d38c49bb62d9c45309f12312d68d08fc5d78488ad7532d5693555648#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":25542666}},"1ef338b0d38c49bb62d9c45309f12312d68d08fc5d78488ad7532d5693555648#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"1fa4ac3fc5fd080396095f517065fcda8e3d3fc8a3151322953afe989ab0fe42#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"5391b4c986e2c6393a699593004face3e4ea35e5c01bc99167c3acc4"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581c5391b4c986e2c6393a699593004face3e4ea35e5c01bc99167c3acc49f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"3263a040d1372a064ae98eaa7d350246cc2897117b7d7fe476cdcd4e86ae9d0f","referenceScript":null,"value":{"5391b4c986e2c6393a699593004face3e4ea35e5c01bc99167c3acc4":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"1fa4ac3fc5fd080396095f517065fcda8e3d3fc8a3151322953afe989ab0fe42#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"20596243dce037185135d398e2c34ed55abf7d8aac92a707fe733c0ab80998ea#1":{"address":"addr_test1vz5wrrhljfvqp93f9hwvpmydt54xad6k3d7uz7fldtjgr6cqktk8m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"240be58f60c313a192e1c6d252d13e48f75c786092ce728601cb36c549198d6c#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a"},{"list":[{"bytes":"a65dd1af5d938e394b05198b97870a33343d3dd2ec8feb8906a2cbbd2cc0ac96"},{"bytes":"32bce7668b68bcc66750963217e50244fb7a9495053fb787fd5480aa1e509366"},{"bytes":"935df47d312772df391cc7a934702e13e9f4263c611cc275b93f76ec6d31d02c"},{"bytes":"9e0e0a73d7e878d6719d374c667e99ff2b5e933130d4ca67a335cca06443a34d"},{"bytes":"8c69cf05a794ae72151dbd7522d9398daf30b14d0a8f9972e31f76fb14b0b4f1"},{"bytes":"6865953562830aee1faaedf2353aed54519743d9c427f2f75a89a8485e3d60ee"}]},{"constructor":0,"fields":[{"int":300000}]},{"int":0},{"bytes":"a07f59abf3032d36ac322e6b423729d70799262df3e6361cfbc885a8ba61db8e"}]}]},"inlineDatumRaw":"d87a9fd8799f581c2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a9f5820a65dd1af5d938e394b05198b97870a33343d3dd2ec8feb8906a2cbbd2cc0ac96582032bce7668b68bcc66750963217e50244fb7a9495053fb787fd5480aa1e5093665820935df47d312772df391cc7a934702e13e9f4263c611cc275b93f76ec6d31d02c58209e0e0a73d7e878d6719d374c667e99ff2b5e933130d4ca67a335cca06443a34d58208c69cf05a794ae72151dbd7522d9398daf30b14d0a8f9972e31f76fb14b0b4f158206865953562830aee1faaedf2353aed54519743d9c427f2f75a89a8485e3d60eeffd8799f1a000493e0ff005820a07f59abf3032d36ac322e6b423729d70799262df3e6361cfbc885a8ba61db8effff","inlineDatumhash":"c7acd5e1756a50824e847166be8291810d6a3a00a85b9ab3916db538f0e6ad7d","referenceScript":null,"value":{"2751338212b299139df293a3dd9c9f32a19774bc14ae9031c1de636a":{"4879647261486561645631":1,"4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b41":1,"5ecf89bf0e00eaf8667d9ff9bb92c6faac2fbcc6ff71ce9a8d6c5033":1,"6cd6343f508777d0da115ba6e42b7c68f83d41a2f17d772b2f849432":1,"96c6d56eb5b0b80595fa11df3cccbdf8523a0f6d869ea162be0fde23":1,"da0efb26cb9b09c64a3725704609d3ede6d72dc985d10e10dc6a19c7":1,"db71b965a22cf2849a903f81c5aba1ccd9108f2b636477351c5b13c2":1},"lovelace":13012593980}},"240be58f60c313a192e1c6d252d13e48f75c786092ce728601cb36c549198d6c#1":{"address":"addr_test1vpkdvdpl2zrh05x6z9d6dept0350s02p5tch6aet97zfgvsjgutah","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9993878957}},"29921589c7e54e8c43559d5181465abd8ef3baa50e085717eafd78124e6ff327#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763169578118},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"bb20626bc0ec1528216c55dc67f7e305665caa567f40f3977d1b3dd7b41e67de"},{"int":1}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a090d4824d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a8518ac869fd8799fd8799f5820bb20626bc0ec1528216c55dc67f7e305665caa567f40f3977d1b3dd7b41e67de01ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58233c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a090d4824d87980d87a80ffffffffff","inlineDatumhash":"49efcff6d5556ea29af4fb697d99af405caf17779134b5fce95f41e97dcf0e3c","referenceScript":null,"value":{"lovelace":151865380}},"29921589c7e54e8c43559d5181465abd8ef3baa50e085717eafd78124e6ff327#1":{"address":"addr_test1vpwptfr5hy0lu3etrrm5f2az39zrxyn9e6ucaa4e4lnnu3gcju78e","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9958536387}},"2a3216b2d30f38d8dbd242ade364290be69f878975817e22530407744349d446#1":{"address":"addr_test1vztvd4twkkctspv4lgga70xvhhu9yws0dkrfagtzhc8augc8jsq6v","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9967570604}},"2be1deccd74ee8d9e027ac4401da21d5d769a990a6a43b34e6e9bbbd245390df#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"22c3e44e2a4bdafa2c656082ff02acb0790428ec7c5bddc91e66644f"},{"list":[{"bytes":"19195f576bb0a022871f78e964d9218869814e3009f6bf9f4f14dac0747f18e7"},{"bytes":"1a76ea7e94e7db2b2aa00bf932b2e93d2999e3e7037004ef1223c648e7c08b8e"}]},{"constructor":0,"fields":[{"int":60000}]},{"int":0},{"bytes":"8c1d8b6585e3bfd4f035e3b4e1dec3c492be9e463e359510927e362d50988ebc"}]}]},"inlineDatumRaw":"d87a9fd8799f581c22c3e44e2a4bdafa2c656082ff02acb0790428ec7c5bddc91e66644f9f582019195f576bb0a022871f78e964d9218869814e3009f6bf9f4f14dac0747f18e758201a76ea7e94e7db2b2aa00bf932b2e93d2999e3e7037004ef1223c648e7c08b8effd8799f19ea60ff0058208c1d8b6585e3bfd4f035e3b4e1dec3c492be9e463e359510927e362d50988ebcffff","inlineDatumhash":"f6769ce04f74819be81fec8489f86508bfd95e10459f3b591c3d80f597b4ff14","referenceScript":null,"value":{"22c3e44e2a4bdafa2c656082ff02acb0790428ec7c5bddc91e66644f":{"4879647261486561645631":1,"54208f029f1154b0abe58bda19756316ac24db456c74292c950132b9":1,"922bc8300929661ee755d15ea62151fa6ec8b993d2c94bb36c94d3a5":1},"lovelace":64318620}},"2be1deccd74ee8d9e027ac4401da21d5d769a990a6a43b34e6e9bbbd245390df#1":{"address":"addr_test1vp2zprcznug4fv9tuk9a5xt4vvt2cfxmg4k8g2fvj5qn9wgxnv0rk","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19547195}},"2d5ec056e24097a742c4257a321f27e083632a0acf5155ccc8233d3e93052068#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":42307383}},"2d5ec056e24097a742c4257a321f27e083632a0acf5155ccc8233d3e93052068#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"2da9d97dbf90341472f25a38c72b4bb3b90ec9b62fb1f36ef1f9a4578c6a7fb1#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763179511831},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"9bff9c627ead684191e26e3cf5105775ef59f2c791477d129211771f9f46f05a"},{"int":1}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a025f9f43d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a85b040179fd8799fd8799f58209bff9c627ead684191e26e3cf5105775ef59f2c791477d1241a1028ac056[764]: 9211771f9f46f05a01ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58233c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a025f9f43d87980d87a80ffffffffff","inlineDatumhash":"c73c71656b451619ef41deb0573ce14d3ec444d33760c5a4304707058030d075","referenceScript":null,"value":{"lovelace":39821123}},"2da9d97dbf90341472f25a38c72b4bb3b90ec9b62fb1f36ef1f9a4578c6a7fb1#1":{"address":"addr_test1vpwptfr5hy0lu3etrrm5f2az39zrxyn9e6ucaa4e4lnnu3gcju78e","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9953647172}},"2de2c5e3fcc260eeb98cc0df2af1cd1eb7b3108cd58a07ac80f9dad3bc3ddb0e#1":{"address":"addr_test1vzz8fc35mtxqmhtn4xa2eylmd8hx4frfda46cn4vspyeygsgevl35","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"3157700ea8f6dbe120fd4fc82425f537155649b69b9dcac39f78255af5dd006b#1":{"address":"addr_test1vqf09e90hzd0m584692ku37gwyvvpv9gsrnp2wgae66j9fgjswpqn","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"3397a73228eae257370df8aa8289d74ec3038faa4e5308347d8752a299351397#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":26798848}},"3397a73228eae257370df8aa8289d74ec3038faa4e5308347d8752a299351397#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#0":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#1":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#10":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#11":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#12":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#13":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#14":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#15":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#16":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#17":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#18":{"address":"addr_test1vq40pslmsyfpxu8qjgxnsd7wp9a2zws8p66myvatr7t58xsyqac97","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19426419}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#2":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#3":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#4":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#5":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#6":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#7":{"address":"addr_test1vqy8dfrjkf7d8d7557rgewduv87uhc8wph32g48kmqslhlgfe0yq5","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":14000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#8":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"347d241574ea9e361d94babafd1d9aef2aa800b077fa869fde4afc19653610d4#9":{"address":"addr_test1vrc68u94dq6h8u9ykkdk3vvf3hpfqjdasm2d6psnwh8gxmc6emrja","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"38e24bb01bd5f1275699a3ede314fd4541a2e58cf69ac766c8793fc3ac900c33#1":{"address":"addr_test1vp8m20m650s8u0em4pgka569zx0fssvzywzuqvrnzysfksgxeq2s2","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":16393118400}},"3a686d32ed9da6ab510c22f12e8d2464a075a5e725bb6b2af575425a431633c7#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":0,"fields":[{"constructor":0,"fields":[{"int":600000}]},{"list":[{"bytes":"be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb08813"},{"bytes":"be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb08813"},{"bytes":"eb2a2c69314c3969e25235bfffceeaa3def3a4f54e91e73f946582ccb2345eee"}]},{"bytes":"d185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f"},{"constructor":0,"fields":[{"bytes":"aee27084568179e44effc70a32087b47e6f7e5f9383f2e0163dbc4d437ad48d5"},{"int":3}]}]},"inlineDatumRaw":"d8799fd8799f1a000927c0ff9f5820be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb088135820be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb088135820eb2a2c69314c3969e25235bfffceeaa3def3a4f54e91e73f946582ccb2345eeeff581cd185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8fd8799f5820aee27084568179e44effc70a32087b47e6f7e5f9383f2e0163dbc4d437ad48d503ffff","inlineDatumhash":"78164dce62a98dd3b6e38b99f5de46c082c8215a6650707928cf1bf88a79c7da","referenceScript":null,"value":{"d185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f":{"4879647261486561645631":1},"lovelace":1887780}},"3a686d32ed9da6ab510c22f12e8d2464a075a5e725bb6b2af575425a431633c7#1":{"address":"addr_test1wry2zqd9ezkys94smn44nn33ls393cu8m6pg7q5kr5hjq3g2tzley","datum":null,"inlineDatum":{"bytes":"d185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f"},"inlineDatumRaw":"581cd185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f","inlineDatumhash":"4acca1d985c0c35ecbea59f2ff2b71e215135b04642de2cdc20743750fdff8f4","referenceScript":null,"value":{"d185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f":{"5c15a474b91ffe472b18f744aba28944331265ceb98ef6b9afe73e45":1},"lovelace":1293000}},"3a686d32ed9da6ab510c22f12e8d2464a075a5e725bb6b2af575425a431633c7#2":{"address":"addr_test1wry2zqd9ezkys94smn44nn33ls393cu8m6pg7q5kr5hjq3g2tzley","datum":null,"inlineDatum":{"bytes":"d185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f"},"inlineDatumRaw":"581cd185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f","inlineDatumhash":"4acca1d985c0c35ecbea59f2ff2b71e215135b04642de2cdc20743750fdff8f4","referenceScript":null,"value":{"d185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f":{"5c15a474b91ffe472b18f744aba28944331265ceb98ef6b9afe73e45":1},"lovelace":1293000}},"3a686d32ed9da6ab510c22f12e8d2464a075a5e725bb6b2af575425a431633c7#3":{"address":"addr_test1wry2zqd9ezkys94smn44nn33ls393cu8m6pg7q5kr5hjq3g2tzley","datum":null,"inlineDatum":{"bytes":"d185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f"},"inlineDatumRaw":"581cd185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f","inlineDatumhash":"4acca1d985c0c35ecbea59f2ff2b71e215135b04642de2cdc20743750fdff8f4","referenceScript":null,"value":{"d185a4fa04b3e37c9635936c3f71fd1a8cb772c306e9288f92d50b8f":{"5975ef036a321619a2cb422cd4e2c8a31ae11c7cf222237749ab0027":1},"lovelace":1293000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#0":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#1":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#10":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#11":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":40000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#12":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#13":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#14":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#15":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#16":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#17":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#18":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#19":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#2":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#20":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#21":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#22":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#23":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#24":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#25":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#26":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#27":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#28":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#29":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#3":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#30":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#31":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#32":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#33":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#34":{"address":"addr_test1vq5sw6pc7rfnml5cw7asenmd6r86d4crq9zc9pkthq5npdsm3xk5c","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19400239}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#4":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#5":{"address":"addr_test1vp96emay9navpkr662fypdgtmpcpchsqr99cxpdavgq67pcl2dsx2","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":8000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#6":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#7":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#8":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"3c68ab84284e8f063d1f81bc38897a38a21ea221c9005120a6c01636f751eb0d#9":{"address":"addr_test1vpf9nyczwfw0zyhu5n4x0xau4tquzsxnguklr5ccqy5wsysd3xu8z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"45e7a28fdcbbceb7b77e128f2bbdad3c54dad1b1c06913970662577c51b537ce#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"4c69f4aa0e163dc930431a7b0a02df0a435c87567ac886931cff1fa0"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581c4c69f4aa0e163dc930431a7b0a02df0a435c87567ac886931cff1fa09f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"63cf5bb48a4b42db25c046e772e564a44ab6fff650e43c986188d96b4c058dfe","referenceScript":null,"value":{"4c69f4aa0e163dc930431a7b0a02df0a435c87567ac886931cff1fa0":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"45e7a28fdcbbceb7b77e128f2bbdad3c54dad1b1c06913970662577c51b537ce#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"47e56bb5680edbf7b8e5b85c5d3e8b40fc9468d4481830f440b87fe6548f5c4a#1":{"address":"addr_test1vrdqa7exewdsn3j2xujhq3sf60k7d4edexzazrssm34pn3c722qsw","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9928156294}},"4ac78faf1da6a0c658b3b490e4d7eee95d67a2e9699c58c6419acb79a8d6f287#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"28dab0e59ce37d84bd70c07f0b8bef0c900e870ee21a9546c2804dc1"},{"list":[{"bytes":"09c478b509c39e7703a341d8c31a89347ae2c4a33f53bee2ae0f865cd4841f52"},{"bytes":"73a9ec5e0210198a642c69f233fecde6f04865c1179c648e3a6b16f43d1a9939"}]},{"constructor":0,"fields":[{"int":60000}]},{"int":0},{"bytes":"87abc24742e36a0af37eba06a2c6c5b60d7a341472efcffb7328803b1cc5562f"}]}]},"inlineDatumRaw":"d87a9fd8799f581c28dab0e59ce37d84bd70c07f0b8bef0c900e870ee21a9546c2804dc19f582009c478b509c39e7703a341d8c31a89347ae2c4a33f53bee2ae0f865cd4841f52582073a9ec5e0210198a642c69f233fecde6f04865c1179c648e3a6b16f43d1a9939ffd8799f19ea60ff00582087abc24742e36a0af37eba06a2c6c5b60d7a341472efcffb7328803b1cc5562fffff","inlineDatumhash":"3a0456c1a5c117f079b7e25411a709b1c38021f2b8bfbc93b87ebbb53f2a6d37","referenceScript":null,"value":{"28dab0e59ce37d84bd70c07f0b8bef0c900e870ee21a9546c2804dc1":{"42621aad9c1b78b81719631d4d8a656e8ab6ae5042759503179bb054":1,"4879647261486561645631":1,"8474e234dacc0ddd73a9baac93fb69ee6aa4696f6bac4eac80499222":1},"lovelace":64318620}},"4ac78faf1da6a0c658b3b490e4d7eee95d67a2e9699c58c6419acb79a8d6f287#1":{"address":"addr_test1vppxyx4dnsdh3wqhr9336nv2v4hg4d4w2pp8t9grz7dmq4qatrxlh","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19547195}},"4afd02469870ffefbf459b8babcd041f6bc951a50508048e42a9e8fc83c80a2f#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":0,"fields":[{"constructor":0,"fields":[{"int":600000}]},{"list":[{"bytes":"be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb08813"},{"bytes":"eb2a2c69314c3969e25235bfffceeaa3def3a4f54e91e73f946582ccb2345eee"}]},{"bytes":"3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550"},{"constructor":0,"fields":[{"bytes":"fd9a0a48eb5e25e9d903b9bc94f3e20e607ed5eb9e642c32249bdf48d755989e"},{"int":4}]}]},"inlineDatumRaw":"d8799fd8799f1a000927c0ff9f5820be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb088135820eb2a2c69314c3969e25235bfffceeaa3def3a4f54e91e73f946582ccb2345eeeff581c3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550d8799f5820fd9a0a48eb5e25e9d903b9bc94f3e20e607ed5eb9e642c32249bdf48d755989e04ffff","inlineDatumhash":"6e0ddabdb236d1389819fe8a7a818d1db13e6a537ebbd6e27c8dae98d50d4179","referenceScript":null,"value":{"3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550":{"4879647261486561645631":1},"lovelace":1741240}},"4afd02469870ffefbf459b8babcd041f6bc951a50508048e42a9e8fc83c80a2f#2":{"address":"addr_test1wry2zqd9ezkys94smn44nn33ls393cu8m6pg7q5kr5hjq3g2tzley","datum":null,"inlineDatum":{"bytes":"3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550"},"inlineDatumRaw":"581c3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550","inlineDatumhash":"a40d0066f4bd23d39160087a003de6a054966f4a19314316c5fc1c22c6188b91","referenceScript":null,"value":{"3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550":{"5975ef036a321619a2cb422cd4e2c8a31ae11c7cf222237749ab0027":1},"lovelace":1293000}},"4d5938e8d57e8aa2809c94a97e0d897eab9f596fcabe51ac238524978ddf43d1#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":38373607}},"4d5938e8d57e8aa2809c94a97e0d897eab9f596fcabe51ac238524978ddf43d1#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"4e0168ddc8f0e82269b8d80c377decd8491c07ad14edfbc61f9a9784394f61ac#1":{"address":"addr_test1vrdhrwt95gk09py6jqlcr3dt58xdjyy09d3kgae4r3d38ssvwn7g7","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9961412193}},"4f7c061512a05785dcd63ddf5b25b06d2943c77e279d5827617a219890ef70c5#1":{"address":"addr_test1vphzqnm87pk5eajsa2wy95g7c7xwyn254h8wgapshvyvkhsm3p3hz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"4ff9baf288f9717cbbea4047b867c8be6ce67db80180183590285ca00c2a1835#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763172236805},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"2fd641f6cf778cf2cf15c51d099ee4a39f37446f4f20bbab756ad7245ce291ab"},{"int":1}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a025f9f43d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a85413e059fd8799fd8799f58202fd641f6cf778cf2cf15c51d099ee4a39f37446f4f20bbab756ad7245ce291ab01ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58233c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a025f9f43d87980d87a80ffffffffff","inlineDatumhash":"9e229594f8ff762681cda4e8b5274411f18aea2bf4f3b26753094323a0f6afae","referenceScript":null,"value":{"lovelace":39821123}},"57096f8ac343a33d58475aac938676aca43bacbb00527cc8f2002d0eb60b5cae#1":{"address":"addr_test1vp6y3efcum89cvq9jdnwytxz7mm5dfwhquafqk4jakjt8yspws7gw","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"5966e68f36467afa297942158f79b1883b96ce3eef3aa98ff4837faff6ac5a8b#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"11119ab8ed5c35aa217b7e65b73e50b50fc5f538e6bcec570cf19842"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581c11119ab8ed5c35aa217b7e65b73e50b50fc5f538e6bcec570cf198429f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"aaaad32e9772ba172511457338f93159b9ee65c29cd964f3b28273264408d846","referenceScript":null,"value":{"11119ab8ed5c35aa217b7e65b73e50b50fc5f538e6bcec570cf19842":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"5966e68f36467afa297942158f79b1883b96ce3eef3aa98ff4837faff6ac5a8b#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"67b458bba5bc680a41875366405cac4e59d4e0036f2de474f6c73ef2f2a693c6#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763234461154},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"0326bc513192ef87679dd3d522460572bfa82779630ddb8d3c449e175f00eab2"},{"int":1}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a024da322d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a88f6b5e29fd8799fd8799f58200326bc513192ef87679dd3d522460572bfa82779630ddb8d3c449e175f00eab201ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58233c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a024da322d87980d87a80ffffffffff","inlineDatumhash":"220dc1458c63dba214348cbc76ca209157bafe1770fda9e810cae3d28a6b0077","referenceScript":null,"value":{"lovelace":38642466}},"67b458bba5bc680a41875366405cac4e59d4e0036f2de474f6c73ef2f2a693c6#1":{"address":"addr_test1vpwptfr5hy0lu3etrrm5f2az39zrxyn9e6ucaa4e4lnnu3gcju78e","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9951297525}},"685a12c9eb50dc8456b49dad8216e199cb6ed258f34d681f8aaa2a9cfe3faf03#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"24c7b2462aaac95145ad00b7b35007abc9bf6724d7e1e96b138e2b53"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581c24c7b2462aaac95145ad00b7b35007abc9bf6724d7e1e96b138e2b539f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"c2596c7428052aadcfeb3ba34d7718a1062e824c5b8c5ddcfcfa7bdfc415a4ba","referenceScript":null,"value":{"24c7b2462aaac95145ad00b7b35007abc9bf6724d7e1e96b138e2b53":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"685a12c9eb50dc8456b49dad8216e199cb6ed258f34d681f8aaa2a9cfe3faf03#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"6d684ffe2f204fab805d85f4d875a189d784d19a2017870261e5f172e7a027d3#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":12610702}},"6d684ffe2f204fab805d85f4d875a189d784d19a2017870261e5f172e7a027d3#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#0":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#1":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#10":{"address":"addr_test1vqftuxy5e7zag7s76jxhxu4cs8ddxtw0xpd6xmyspftx4pcsjml0w","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19439443}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#2":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#3":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#4":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#5":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#6":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#7":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#8":{"address":"addr_test1vzs0nvgpljpkruhw0z68sgns5gac4p2nn5chtvtj80m2wfstnsr3m","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"6f3f29525782c574ec860a64eeb2ac697ef0c38dba475ca350e7235c7e15ccab#9":{"address":"addr_test1vqtp8vrd8fzalerqakwr04tk7wskpl3hgc5plxp4jc5zgac26h7mh","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":22000000}},"73e48c0afba8aaa9fcb42cb1bb089c9d3589a85c48b9b6b46e324d8f6e28c7ae#1":{"address":"addr_test1vpfsdk36hpge96grw776l85rq2fpef0nu3gcfhf0dc0yegg8gcfsy","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":98081893}},"76fe2186c335e51dbe1170074245479020f9a8c0e3b1ecd3ecab872c2cd83d43#1":{"address":"addr_test1vpun6pmglvvk98uz92rqnzrag6n497zp7kdxvgvrd6xjtaqya3h3k","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"7a6f5cc0e5621f925e6a63fe09e14bab680df5513ab1b92e7eb92d18dc4766ab#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"df94cf6d748c13776dd478469a27f32b7fbad0e52d5fad960c61c76d"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581cdf94cf6d748c13776dd478469a27f32b7fbad0e52d5fad960c61c76d9f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"3df42ba839d25b9d5d16dc0de2ffae97c848b6f57c98d04481933016b6a08272","referenceScript":null,"value":{"df94cf6d748c13776dd478469a27f32b7fbad0e52d5fad960c61c76d":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"7a6f5cc0e5621f925e6a63fe09e14bab680df5513ab1b92e7eb92d18dc4766ab#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"7e08d21a9a36d81bd342e023d3bdb94872f757b260e7adf5a39cbe0842bc83e4#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":47990238}},"7e08d21a9a36d81bd342e023d3bdb94872f757b260e7adf5a39cbe0842bc83e4#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"8ac3ac48f0f649c753766ff8794aa5470009507d41883a3302634caecca5ead6#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763160998449},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"ce9a8df77f11d74180f3da1704706f64a509a751d3f6395d9aae917e87b4e698"},{"int":0}]},{"bytes":"d8799fd8799fd8799f581c5975ef036a321619a2cb422cd4e2c8a31ae11c7cf222237749ab0027ffd87a80ffa140a1401a004c4b40d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a8495c2319fd8799fd8799f5820ce9a8df77f11d74180f3da1704706f64a509a751d3f6395d9aae917e87b4e69800ff583cd8799fd8799fd8799f581c5975ef036a321619a2cb422cd4e2c8a31ae11c7cf222237749ab0027ffd87a80ffa140a1401a004c4b40d87980d87a80ffffffff","inlineDatumhash":"2adc55c841b2265880b135eb02ec2bcc60ab7ae4a755dc07647b793ff28c8381","referenceScript":null,"value":{"lovelace":5000000}},"8ac3ac48f0f649c753766ff8794aa5470009507d41883a3302634caecca5ead6#1":{"address":"addr_test1vpvhtmcrdgepvxdzedpze48zez334cgu0nezygmhfx4sqfch2cqqz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":25794791}},"8cdc16f13adb3bc046c6e3fa02a8bee6a29db62a681e9c27be92e4c7abbe6d20#0":{"address":"addr_test1vzle8p67pwx5868240vyxkgnvwukrk9gu40sxswjjjahd5g5ug2er","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"8cdc16f13adb3bc046c6e3fa02a8bee6a29db62a681e9c27be92e4c7abbe6d20#1":{"address":"addr_test1vzle8p67pwx5868240vyxkgnvwukrk9gu40sxswjjjahd5g5ug2er","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"8cdc16f13adb3bc046c6e3fa02a8bee6a29db62a681e9c27be92e4c7abbe6d20#2":{"address":"addr_test1vz6h6pa7qxp53mqaxzk0gj5q3fvgsf2zlr7sn3p6ph89ssctf6juc","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28000000}},"8cdc16f13adb3bc046c6e3fa02a8bee6a29db62a681e9c27be92e4c7abbe6d20#3":{"address":"addr_test1vzle8p67pwx5868240vyxkgnvwukrk9gu40sxswjjjahd5g5ug2er","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"8cdc16f13adb3bc046c6e3fa02a8bee6a29db62a681e9c27be92e4c7abbe6d20#4":{"address":"addr_test1vzs6470ds6y3ecjzuyxkewgla9y2penrvz2y8x8tnu979rsakyhry","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19449211}},"8f36b43e9106262bd9c682dcd738399b2f2b7975e771bb168251def48b147941#1":{"address":"addr_test1vqluwjpud6d8g362yp4xr8gangek95wl7emz5a4g9dfd79scwnccp","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"934f7be5a9d0f60e9b36a9d8ea4448995ab49ed8665bf6a93dfa8eb49c352f09#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763174029780},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"e215ecd3f7c37d73dc3c04a66f3021ed38f6bcca5c606af228c6f62971ab9270"},{"int":1}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a025f9f43d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a855c99d49fd8799fd8799f5820e215ecd3f7c37d73dc3c04a66f3021ed38f6bcca5c606af228c6f62971ab927001ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58233c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a025f9f43d87980d87a80ffffffffff","inlineDatumhash":"0ce3121e54674c8f27eae2b56fd316381665da6f7850dd48cc41fe1a3362bea8","referenceScript":null,"value":{"lovelace":39821123}},"934f7be5a9d0f60e9b36a9d8ea4448995ab49ed8665bf6a93dfa8eb49c352f09#1":{"address":"addr_test1vpwptfr5hy0lu3etrrm5f2az39zrxyn9e6ucaa4e4lnnu3gcju78e","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9955996819}},"94fe0a917aa87b8ecac30e11e04a492941cff1da19a69bfaff28a241a2671004#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":0,"fields":[{"constructor":0,"fields":[{"int":3000}]},{"list":[{"bytes":"b37aabd81024c043f53a069c91e51a5b52e4ea399ae17ee1fe3cb9c44db707eb"}]},{"bytes":"8225a09fbb45e7807e2bde2af52022b44bdfefc5d5308a4b2f37a25b"},{"constructor":0,"fields":[{"bytes":"3a3b96383d15725d1d39394b39fd0297dea500fd5ae2853ca133376a27c3e4f6"},{"int":0}]}]},"inlineDatumRaw":"d8799fd8799f190bb8ff9f5820b37aabd81024c043f53a069c91e51a5b52e4ea399ae17ee1fe3cb9c44db707ebff581c8225a09fbb45e7807e2bde2af52022b44bdfefc5d5308a4b2f37a25bd8799f58203a3b96383d15725d1d39394b39fd0297dea500fd5ae2853ca133376a27c3e4f600ffff","inlineDatumhash":"9632a451c72611b42db5fb4cba3c60fca3ec4337160440dae7ca6d35d36579cc","referenceScript":null,"value":{"8225a09fbb45e7807e2bde2af52022b44bdfefc5d5308a4b2f37a25b":{"4879647261486561645631":1},"lovelace":1586080}},"94fe0a917aa87b8ecac30e11e04a492941cff1da19a69bfaff28a241a2671004#1":{"address":"addr_test1wry2zqd9ezkys94smn44nn33ls393cu8m6pg7q5kr5hjq3g2tzley","datum":null,"inlineDatum":{"bytes":"8225a09fbb45e7807e2bde2af52022b44bdfefc5d5308a4b2f37a25b"},"inlineDatumRaw":"581c8225a09fbb45e7807e2bde2af52022b44bdfefc5d5308a4b2f37a25b","inlineDatumhash":"2fb78d2ab99f5f954910d87440b0cb12aa694186d97de8e4ce3d53d5d3de3c88","referenceScript":null,"value":{"8225a09fbb45e7807e2bde2af52022b44bdfefc5d5308a4b2f37a25b":{"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":1293000}},"9cf778b022c93a564c9956c46e9c5fd6618a5486c0a3448a64463e93a6c39148#1":{"address":"addr_test1vqq7s2qsftqpk5c87427zpflcvzzz9zcj736t24fy8qktds263v6q","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"9e85043514dbdfd2b007e74f6bf1905cdd1acb85dfae819d4892e376291aa044#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"361551687585878d505633f900307970c7b73128a6976c399dd94a01"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581c361551687585878d505633f900307970c7b73128a6976c399dd94a019f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"4bb564d458319b1663a3ecb5fbcfa975ef683b280fc525ce4594c1154920780a","referenceScript":null,"value":{"361551687585878d505633f900307970c7b73128a6976c399dd94a01":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"9e85043514dbdfd2b007e74f6bf1905cdd1acb85dfae819d4892e376291aa044#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"a02bc677c3e2f881362d83aa7378d505cba0cf17182a3dd67dc5ebc736f09c89#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"bc98fd9be3f5f810b073a6758ba590cb13d20d7ae43247dfd64ef6d9"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581cbc98fd9be3f5f810b073a6758ba590cb13d20d7ae43247dfd64ef6d99f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"eba7ccabd3f02ab98e23577e3725c1fe6b4c8beda3f10e351155dad06dc995eb","referenceScript":null,"value":{"bc98fd9be3f5f810b073a6758ba590cb13d20d7ae43247dfd64ef6d9":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"a02bc677c3e2f881362d83aa7378d505cba0cf17182a3dd67dc5ebc736f09c89#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"a4cc5aee87a9c211d7cff9d74b1d9241e38e315a2a80b170f463fa1289cc1e40#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":45518095}},"a4cc5aee87a9c211d7cff9d74b1d9241e38e315a2a80b170f463fa1289cc1e40#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"a6254c9f833ee3a3407f9a3b8f3aa42410b734cb1b43f230cf8eacfc4921becd#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":11629852}},"a6254c9f833ee3a3407f9a3b8f3aa42410b734cb1b43f230cf8eacfc4921becd#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9984498116}},"a98f011752171c39c24df2edba85b630da9f7cb328e5295b32495711e82f1dea#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"2b75e5652a0711d9e713958eae54f3cc81bc4088ff57c6443d60481f"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581c2b75e5652a0711d9e713958eae54f3cc81bc4088ff57c6443d60481f9f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"2ff06f5fec919acdf171fd634c79d8af27b979b683d745808f832134e7e14062","referenceScript":null,"value":{"2b75e5652a0711d9e713958eae54f3cc81bc4088ff57c6443d60481f":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"a98f011752171c39c24df2edba85b630da9f7cb328e5295b32495711e82f1dea#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"aee27084568179e44effc70a32087b47e6f7e5f9383f2e0163dbc4d437ad48d5#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":0,"fields":[{"constructor":0,"fields":[{"int":600000}]},{"list":[{"bytes":"be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb08813"},{"bytes":"eb2a2c69314c3969e25235bfffceeaa3def3a4f54e91e73f946582ccb2345eee"}]},{"bytes":"2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9"},{"constructor":0,"fields":[{"bytes":"c42a263d26588658af772934bde147dfec5310bf932a4504fdd5d46fc346dca6"},{"int":0}]}]},"inlineDatumRaw":"d8799fd8799f1a0009270ff9f5820be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb088135820eb2a2c69314c3969e25235bfffceeaa3def3a4f54e91e73f946582ccb2345eeeff581c2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9d8799f5820c42a263d26588658af772934bde147dfec5310bf932a4504fdd5d46fc346dca600ffff","inlineDatumhash":"190d7db3027110b820e43a895e87b126f4faf671807207eb62cd70b7ddddab1f","referenceScript":null,"value":{"2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9":{"4879647261486561645631":1},"lovelace":1741240}},"aee27084568179e44effc70a32087b47e6f7e5f9383f2e0163dbc4d437ad48d5#1":{"address":"addr_test1wry2zqd9ezkys94smn44nn33ls393cu8m6pg7q5kr5hjq3g2tzley","datum":null,"inlineDatum":{"bytes":"2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9"},"inlineDatumRaw":"581c2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9","inlineDatumhash":"153a91a55399760c6e7b31261607a9098644b90139f563b4a1744aa38a4860a3","referenceScript":null,"value":{"2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9":{"5c15a474b91ffe472b18f744aba28944331265ceb98ef6b9afe73e45":1},"lovelace":1293000}},"aee27084568179e44effc70a32087b47e6f7e5f9383f2e0163dbc4d437ad48d5#2":{"address":"addr_test1wry2zqd9ezkys94smn44nn33ls393cu8m6pg7q5kr5hjq3g2tzley","datum":null,"inlineDatum":{"bytes":"2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9"},"inlineDatumRaw":"581c2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9","inlineDatumhash":"153a91a55399760c6e7b31261607a9098644b90139f563b4a1744aa38a4860a3","referenceScript":null,"value":{"2f9b279c4044a6a53cb7140df44581a4f21b566f8e4ecfe2c8d11ee9":{"5c15a474b91ffe472b18f744aba28944331265ceb98ef6b9afe73e45":1},"lovelace":1293000}},"b1497e0bbb771e52fa8bf3fe4f9c2d0dc56eed7c4886755e5620e92e291bfd7b#0":{"address":"addr_test1wps5tz7z72tllu7vthm2c7440nhasamrkzmm6u3pg6ssxhqs8uluy","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb08813"},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"3e86699711dfa2f983094e4fc134acc16125151b28fc84b11f07507f063bc02e"},{"int":1}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401b0000000215217eeed87980d87a80ff"}]}]},{"bytes":"3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550"}]},"inlineDatumRaw":"d8799f5820be6dfb1dba07f3bc0df79a0df39c607a2c7d0ecbdf74d3c37f935acb3fb088139fd8799fd8799f58203e86699711dfa2f983094e4fc134acc16125151b28fc84b11f07507f063bc02e01ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58273c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401b0000000215217eeed87980d87a80ffffffff581c3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550ff","inlineDatumhash":"5ad9e5b6d1f7b145616f26a712015de73f0bade5975b0c8d895d94478b414d20","referenceScript":null,"value":{"3bb16768439700872a47788f40d0dec3d8b684982332508076c3d550":{"5c15a474b91ffe472b18f744aba28944331265ceb98ef6b9afe73e45":1},"lovelace":8945744310}},"b1497e0bbb771e52fa8bf3fe4f9c2d0dc56eed7c4886755e5620e92e291bfd7b#1":{"address":"addr_test1vpwptfr5hy0lu3etrrm5f2az39zrxyn9e6ucaa4e4lnnu3gcju78e","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9969440353}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#0":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#1":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#10":{"address":"addr_test1vr3afvs8thj0x5zm0uu59gl8rthmcg7mtff72qw2awg7fwg5r22tv","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19439443}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#2":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#3":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#4":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#5":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#6":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#7":{"address":"addr_test1vpelwj0j9696z6mg50wk40g6k802vcdf8ywl4vyp36qgs5qmcryhc","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":22000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#8":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"b4e7cbe8b0a107f906796452c745129c0b50dfda60c9704a05fc274332dc7867#9":{"address":"addr_test1vqrs9nvr7l67nl9ysjlg325qtqmr04yxt4slslgz96x8x9slg82gu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"b5eb35d5fdd24660d7aae958e6ed4a94a5576d34cfca9562ddab5e726452ddc9#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"d4b669ea90e6bbce14e3f866e3f5466f55ffd376d7f820cfcada54bf"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581cd4b669ea90e6bbce14e3f866e3f5466f55ffd376d7f820cfcada54bf9f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"5237066feac9a6ebb7507870a515a4ce262d769d9e0b35557bbef1ac07b4fadd","referenceScript":null,"value":{"d4b669ea90e6bbce14e3f866e3f5466f55ffd376d7f820cfcada54bf":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"b5eb35d5fdd24660d7aae958e6ed4a94a5576d34cfca9562ddab5e726452ddc9#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"c23ce36a3401b1eec7670c8237cdea1d392fda2d4ba8a7f354da6937be915895#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763409851713},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"383584cbbffe76e435d5dae0a46bf6fee0c35ed0fbb22fd0ad12fab647c59bf3"},{"int":0}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a02faf080d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a936af5419fd8799fd8799f5820383584cbbffe76e435d5dae0a46bf6fee0c35ed0fbb22fd0ad12fab647c59bf300ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58233c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a02faf080d87980d87a80ffffffffff","inlineDatumhash":"d92f6635ec9b865cdb0faddc8755eda5ac37b34b38bca5856b30e7737923251c","referenceScript":null,"value":{"lovelace":50000000}},"c23ce36a3401b1eec7670c8237cdea1d392fda2d4ba8a7f354da6937be915895#1":{"address":"addr_test1vpwptfr5hy0lu3etrrm5f2az39zrxyn9e6ucaa4e4lnnu3gcju78e","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9950197993}},"c2e02efd2c93fdc48325dedf35b325b2fc64805c97ef6dca23758e0e70fcd6ff#1":{"address":"addr_test1vphsnknjsaal6pdl8p486qdmntefevrpey655sh7ghujv4g3ga33z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"c30ff91bcbd09ca2f6eb56b2d2d142d13400eb6e092636dc67167fd24090be1f#1":{"address":"addr_test1vzfzhjpspy5kv8h82hg4af3p28axaj9ej0fvjjandj2d8fgtjn2js","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"c5823c8050fb06f4605f0afe2008c9cedf7dccc4e7e043c764407f86e5b05a41#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"a44e5a7f32980b869cd78cd54ccf976c8be31248d95fc8dc6b939e26"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581ca44e5a7f32980b869cd78cd54ccf976c8be31248d95fc8dc6b939e269f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"577dfdbb6acebe90cceedb7f91538d980bfd36d698ec06dcafc1d04e47bcd5ff","referenceScript":null,"value":{"a44e5a7f32980b869cd78cd54ccf976c8be31248d95fc8dc6b939e26":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"c5823c8050fb06f4605f0afe2008c9cedf7dccc4e7e043c764407f86e5b05a41#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"c6689f2623c6c282ee14c719ce34de8b5853b6584c166e9da6442d85c071eabd#0":{"address":"addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j","datum":null,"inlineDatum":{"constructor":0,"fields":[{"bytes":"0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af796"},{"int":1763411375003},{"list":[{"constructor":0,"fields":[{"constructor":0,"fields":[{"bytes":"32b984340232b02ab52d85a2bad4bd6580b70deaf2fcca38f8e2ea01150e81bb"},{"int":0}]},{"bytes":"d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c3c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a01c9c380d87980d87a80ff"}]}]}]},"inlineDatumRaw":"d8799f581c0ffad7e9080d9c2e48f320dd6354baa1a60d060d65178014580af7961b0000019a9382339b9fd8799fd8799f582032b984340232b02ab52d85a2bad4bd6580b70deaf2fcca38f8e2ea01150e81bb00ff5f5840d8799fd8799fd8799f581c6426234006526ae0f904322aa670325bbc4962adb854bd2897344de3ffd8799fd8799fd8799f581cd9bdec75b5aa93d205cfc5f52c58233c8487efbe7c6436a2082203c8d7f5ffffffffa140a1401a01c9c380d87980d87a80ffffffffff","inlineDatumhash":"6f1bf723f593ab68dda25ec3e3e6bdbf7b4bb24666a6c9119e0e687d2e819a4b","referenceScript":null,"value":{"lovelace":30000000}},"c6689f2623c6c282ee14c719ce34de8b5853b6584c166e9da6442d85c071eabd#1":{"address":"addr_test1vpvhtmcrdgepvxdzedpze48zez334cgu0nezygmhfx4sqfch2cqqz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":9709321032}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#0":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#1":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#10":{"address":"addr_test1vqn2rvws0kh78ntavzq9alwxv73vyac7gvukkztu4cxdhjga40lyq","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19439443}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#2":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#3":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#4":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#5":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#6":{"address":"addr_test1vz9sjcr8ae0x3l7geavaca4w8sg9yt6qgyk2recfj9rgdmgt5cyfl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":22000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#7":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#8":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"c6bd055c48fd833acd3a269a1989961a9fe4ba2d010c8e95909c82fcc0be82f1#9":{"address":"addr_test1vpa2j4hkcaxrqj36thzxvjmn7wefwk6wy44fj4fazruqucs5k92fz","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"cc15dce18a63f460d4c3b478dfa1deb3ff439c23697244b941d5a0ff85f5863e#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"cf32ec5c1746ec3c89c96c7703a6710c21c1a140cfb820a9feaf6a76"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":200000}]},{"int":0},{"bytes":"b81722beb28fc2965a50f8799aa8d0da91d10c542dae5fd0aa470f1f6e2400a3"}]}]},"inlineDatumRaw":"d87a9fd8799f581ccf32ec5c1746ec3c89c96c7703a6710c21c1a140cfb820a9feaf6a769f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a00030d40ff005820b81722beb28fc2965a50f8799aa8d0da91d10c542dae5fd0aa470f1f6e2400a3ffff","inlineDatumhash":"8eb172afc0096ed033d6f5c18ce4b58ca0b689610b894a9230838e05d3891214","referenceScript":null,"value":{"cf32ec5c1746ec3c89c96c7703a6710c21c1a140cfb820a9feaf6a76":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":24850899}},"cc15dce18a63f460d4c3b478dfa1deb3ff439c23697244b941d5a0ff85f5863e#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":45992195}},"cda361ada4c2b0958c571aad10761df4edeaf8d931d4dfeaafbf67f34becb646#1":{"address":"addr_test1vqyh0npmu6mpxxmugcqeyjx6muxjfk53ecet46qvhzndxcqxy2hnu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#0":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#1":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#10":{"address":"addr_test1vr8644006gwmp8pkq6hrrd89082wep2dg9vdq66y9250f4sxv496t","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19439443}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#2":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#3":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#4":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#5":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#6":{"address":"addr_test1vp4nu89dna8865t73xcpsjc27rgehay6pxktuul7vmrrazcc8puqn","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":22000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#7":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#8":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"cf78f5a302930950cce213a8c137c538d5356ed1a9cbe77e627446522a141c55#9":{"address":"addr_test1vzqku5kuhtfdrvw3j02g2cts3a84mdhm5ll5dp56lwkgdmq5ka5xl","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"d7efcdcca2d69142cdb16ba0a86e9757e6f55f02c9ca61fc4394ef262923321b#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30344221}},"d7efcdcca2d69142cdb16ba0a86e9757e6f55f02c9ca61fc4394ef262923321b#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"da17377508a72208a6dc4f24d2261a5c46e1694fccfc10c5a7d135319838ce8f#0":{"address":"addr_test1vqj7544jw2p6q3tddwx89f7u33u87zvfzfkz3u2h6tt848spkjktw","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"da17377508a72208a6dc4f24d2261a5c46e1694fccfc10c5a7d135319838ce8f#1":{"address":"addr_test1vqj7544jw2p6q3tddwx89f7u33u87zvfzfkz3u2h6tt848spkjktw","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"da17377508a72208a6dc4f24d2261a5c46e1694fccfc10c5a7d135319838ce8f#2":{"address":"addr_test1vqj7544jw2p6q3tddwx89f7u33u87zvfzfkz3u2h6tt848spkjktw","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"da17377508a72208a6dc4f24d2261a5c46e1694fccfc10c5a7d135319838ce8f#3":{"address":"addr_test1vqj7544jw2p6q3tddwx89f7u33u87zvfzfkz3u2h6tt848spkjktw","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"da17377508a72208a6dc4f24d2261a5c46e1694fccfc10c5a7d135319838ce8f#4":{"address":"addr_test1vqj7544jw2p6q3tddwx89f7u33u87zvfzfkz3u2h6tt848spkjktw","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"da17377508a72208a6dc4f24d2261a5c46e1694fccfc10c5a7d135319838ce8f#5":{"address":"addr_test1vzw68qrjau0xhwg6qq86j2t0y4hqtgpxa3h42vmu407nrmscfst0t","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":26000000}},"da17377508a72208a6dc4f24d2261a5c46e1694fccfc10c5a7d135319838ce8f#6":{"address":"addr_test1vp50e0fn9mncpfkyxjqh785u8fcecjyuvjwq02h59n54e0cqraa0n","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19445955}},"e66950d90a983ac6a04794ae3fd33b98245acfdeb6b70d46587c8ba8266dac84#1":{"address":"addr_test1vq64urm0xs065xu66k0kwwmh86cey5g8vr7tzetnay5axjgqw4f9y","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":28081893}},"e97c90027c248a6e0f0c3e206668ce6c44dc739e7d55330b71fe3acc4b11a5c8#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"85688878b4945d2705db44b8c601bb5c70e2898330a7162d37d23002"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581c85688878b4945d2705db44b8c601bb5c70e2898330a7162d37d230029f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"7aee21979c3542bc416221bbd9f5e3d723b1ef677dae5364b07edd6bd2e2e17a","referenceScript":null,"value":{"85688878b4945d2705db44b8c601bb5c70e2898330a7162d37d23002":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"e97c90027c248a6e0f0c3e206668ce6c44dc739e7d55330b71fe3acc4b11a5c8#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"e9c52818d19a47d363cb0f6888eb37e37886834041f547ecb7d8fcb9d320b374#0":{"address":"addr_test1vqxlefma4kzfvjff7kq7vwm7llhxt2zlwc97ahvan8agh3gfr80d9","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":100000000}},"e9c52818d19a47d363cb0f6888eb37e37886834041f547ecb7d8fcb9d320b374#1":{"address":"addr_test1vqxlefma4kzfvjff7kq7vwm7llhxt2zlwc97ahvan8agh3gfr80d9","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"e9c52818d19a47d363cb0f6888eb37e37886834041f547ecb7d8fcb9d320b374#2":{"address":"addr_test1vpmwayuezdatm39ltwdfhfj0y7787hsz7t8zt8qzq3uyc2crc93pu","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":99000000}},"e9c52818d19a47d363cb0f6888eb37e37886834041f547ecb7d8fcb9d320b374#3":{"address":"addr_test1vzxwjxg2f0u7uuewnqwxh0h3q3rnsyk03w8hwm766ydvu7g8sqm49","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":89450575}},"ef3835838697131052433eb5c009e0e2da4fa6041331545d4ac3e7d163b046cd#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"bc9161c791a3dd7885b9c3e21b93747b6280a3aa397abdc8ea25f58f"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581cbc9161c791a3dd7885b9c3e21b93747b6280a3aa397abdc8ea25f58f9f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"bbfd9a741a66bc7cc78a913182e0cc7b7952881d75fb92931a17417835410b3d","referenceScript":null,"value":{"bc9161c791a3dd7885b9c3e21b93747b6280a3aa397abdc8ea25f58f":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"ef3835838697131052433eb5c009e0e2da4fa6041331545d4ac3e7d163b046cd#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"ef3c619409fe284d605976a6113a97ea6b1ac19cd26044580c4278be03137ca2#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"6887bbbd265a98f2ccbcf341e4991d2a7c5f6d24f10865a0ecc1a6d9"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581c6887bbbd265a98f2ccbcf341e4991d2a7c5f6d24f10865a0ecc1a6d99f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"c01b451bc91ee9aa4c6f6805bb5ab70a0a2f10e790a3f2846115039b3c35fe4c","referenceScript":null,"value":{"6887bbbd265a98f2ccbcf341e4991d2a7c5f6d24f10865a0ecc1a6d9":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"ef3c619409fe284d605976a6113a97ea6b1ac19cd26044580c4278be03137ca2#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":15992195}},"f21e530f0758fba70b686bcd52dc55a5bf393ae37830c79a945981eb9af0e757#0":{"address":"addr_test1vp5cxztpc6hep9ds7fjgmle3l225tk8ske3rmwr9adu0m6qchmx5z","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":20416534}},"f21e530f0758fba70b686bcd52dc55a5bf393ae37830c79a945981eb9af0e757#1":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":44478251}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#0":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#1":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#10":{"address":"addr_test1vpu5gn5znh6rtewsr4s8qad34evha6m8uq2pdfx9jgnjwdskmtrph","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":19439443}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#2":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#3":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#4":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":30000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#5":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#6":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#7":{"address":"addr_test1vplzruk26paj3kpw6uclqccrq2f0yhs30lxwaj5mpjg6rlqglsqym","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":22000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#8":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"f2928ccff77355d26c6cae3c019b4dfbfcf857ab60305bc2ce1f4b4c19f373d2#9":{"address":"addr_test1vp6xvff4m4gav5tnqk6wmt56k8g3mm8jsqvjnafqvshruagcugmwj","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":1000000}},"f59de50605f088e7b24653e23da11d6107ce598e5c33ba2797d41351b035fd74#1":{"address":"addr_test1vp0vlzdlpcqw47rx0k0lnwujcma2ctaucmlhrn5634k9qvcx3xrlc","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":698388822}},"f81c0c4cca7bc9179da2ed4c2bf04a1f096c3035cbb13bc23c513c2652e30af2#0":{"address":"addr_test1wzs5gta0ym2wcsy79a32dpwp6jynlrttewhhhj6e6mapxsq390hx3","datum":null,"inlineDatum":{"constructor":1,"fields":[{"constructor":0,"fields":[{"bytes":"aa3035c0e730b5c9fe3c1aec56da7d9785b01fb137618819485988a8"},{"list":[{"bytes":"d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4"}]},{"constructor":0,"fields":[{"int":100000}]},{"int":0},{"bytes":"29f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cd"}]}]},"inlineDatumRaw":"d87a9fd8799f581caa3035c0e730b5c9fe3c1aec56da7d9785b01fb137618819485988a89f5820d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4ffd8799f1a000186a0ff00582029f667bd0d86d441dde0f15a397e81c5e6d6f347d61e11b536f04828267ed7cdffff","inlineDatumhash":"a9d36c430997c9707617aa544a72eb975a978b679983d4bc481898007356882c","referenceScript":null,"value":{"aa3035c0e730b5c9fe3c1aec56da7d9785b01fb137618819485988a8":{"4879647261486561645631":1,"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d":1},"lovelace":7887700}},"f81c0c4cca7bc9179da2ed4c2bf04a1f096c30
...

SB on partial fanout

I want to introduce the on-chain check for fanout that uses the accumulator from the datum and a proof from redeemer and see if the overall Head script size fits into the tx size limits.
First I need to add a proof to the fanout redeemer since this one is missing. I'll try to keep individual commits building so we have easier time if there is a need to pick commits.
Actually first I want to make all tests green - there is one red one in hydra-cluster related to tx observation.
So it seems like close tx was not observed in the ChainObserverSpec.
This happens because we failed to construct close. There was an error PlutusFailedUnexpectedly!
The error happens because the input vs. the output value is not matched in our validator.
This is how rendered clost tx looks like (I've grabbed the information from the logs about the tx and utxo and parsed them in our repl):

  "a48f092b640297b12c8816422642e45f370c019e8d59e6b895ebdaf8539a1163"

== INPUTS (2)

91e9fdc8388e72515abf8cd231bd2dbdb55590de934112c7a836f2ce2acc8ffc#0

ShelleyAddress Testnet (ScriptHashObj (ScriptHash \"e93fdca2467a7aac18e88f63b0f474a78abfba366bdd92078d7f8351\")) StakeRefNull
2879080 lovelace
1 ec192f17647ab59e9085f73fb83115a4e0794b565ae464b3e4d574fc.4879647261486561645631
1 ec192f17647ab59e9085f73fb83115a4e0794b565ae464b3e4d574fc.f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d
TxOutDatumInline [1,[[0,[\"0xec192f17647ab59e9085f73fb83115a4e0794b565ae464b3e4d574fc\",[\"0xd5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4\"],[0,[4000]],1,\"0xe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\"]]]]
ReferenceScriptNone

- 91e9fdc8388e72515abf8cd231bd2dbdb55590de934112c7a836f2ce2acc8ffc#3
ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = \"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d\"})) StakeRefNull
89466135 lovelace
TxOutDatumNone
ReferenceScriptNone

== COLLATERAL INPUTS (1)
- 91e9fdc8388e72515abf8cd231bd2dbdb55590de934112c7a836f2ce2acc8ffc#3
ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = \"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d\"})) StakeRefNull
89466135 lovelace 
TxOutDatumNone
ReferenceScriptNone

== REFERENCE INPUTS (1)
- 7ed1031b2c155f4aad7833deacf3dad159e1476c25ec9f3d4011a0b3953b2725#0

== OUTPUTS (2)

Total number of assets: 3
- ShelleyAddress Testnet (ScriptHashObj (ScriptHash \"e93fdca2467a7aac18e88f63b0f474a78abfba366bdd92078d7f8351\")) StakeRefNull
 2913560 lovelace
 1 ec192f17647ab59e9085f73fb83115a4e0794b565ae464b3e4d574fc.4879647261486561645631
 1 ec192f17647ab59e9085f73fb83115a4e0794b565ae464b3e4d574fc.f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d
 TxOutDatumInline [2,[[0,[\"0xec192f17647ab59e9085f73fb83115a4e0794b565ae464b3e4d574fc\",[\"0xd5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4\"],[0,[4000]],1,1,\"0xe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\",\"0xe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\",\"0xe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\",[],1764154735100,\"0xa2583cd8799fd8799fd8799f581c6356fd332b8162632bb855adc5f2aa87d25f092574191e1ec0fb0245ffd87a80ffa140a1401a001e8480d87980d87a80ff82581cbd16942b519de5e15b1e29e68a71f301d78e77c1f3699d71a35d9b2a01583cd8799fd8799fd8799f581c6356fd332b8162632bb855adc5f2aa87d25f092574191e1ec0fb0245ffd87a80ffa140a1401a007a1200d87980d87a80ff82581cfc3a96525445d35e5d32e6c2d471f6c72ef3bdbe759a5824fad58a2801\"]]]]

 - ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = \"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d\"})) StakeRefNull
 87467537 lovelace
 TxOutDatumNone

 == TOTAL COLLATERAL

 TxTotalCollateralNone

 == RETURN COLLATERAL
 TxReturnCollateralNone

 == FEE
 TxFeeExplicit ShelleyBasedEraConway (Coin 1964118)

== VALIDITY
TxValidityLowerBound AllegraEraOnwardsConway (SlotNo 51)
TxValidityUpperBound ShelleyBasedEraConway (Just (SlotNo 91))

== MINT/BURN
0 lovelace

== SCRIPTS (0)
Total size (bytes):  0
== DATUMS (0)
== REDEEMERS (1)
- ConwaySpending (AsIx {unAsIx = 0}) ( cpu = 10000000000, mem = 14000000 ) 
[3,[[3,[[\"0x75c40e2a53894901b3069f9d3f049a61911ffb81ba1d6f30044b83c085e9f30a7bd5f5c6bbe2dd9c4600c4fe2e060a2f0f14a1d9503d59a36e73993a86876902\"],\"0x48d2475e74478fc71998597d6233c62c4a6f8a3a8a7b067559c9bd19da775261\"]]]]

== REQUIRED SIGNERS
- \"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d\"

== METADATA
TxMetadataInEra ShelleyBasedEraConway (TxMetadata {unTxMetadata = fromList [(55555,TxMetaText \"HydraV1/CloseTx\")]})"

So the input lovelace is 2879080 (a part from collateral input) but somehow in the output we got 2913560 lovelace. Difference is 34480 lovelace
After printing the tx before it goes into our wallet for balancing I notice the input vs. output value matches but the wallet is adding more inputs to cover the fee so that is where we see the discrepancy. We end up with the Head output slightly bigger than the Head input.
How are other tests that involve closing work then?
Continued debugging to find where is the difference between the chain observer test that does close and all the rest of the tests that also involve closing.
What was problematic was that in only this one test we tried to decommit partial amount - all of the other decommit tests were decommitting the whole UTxO but here we tried to only take out 2 ada out of possible 10 in the Head.
After using the complete amount the test works just fine. Question is why did this test work before? I understand that decommitting partially is not advisable since we always mention that the complete UTxO should be taken out but in any case how/why is this triggering errors on close tx?

2025-11-13

SB on debugging hydra-node memory issues

We have a user complaint regarding memory usage in hydra-node and also Heap overflow issue on state replay.
In this stress testing the client is doing they are limiting the hydra-node processes to the following specs:

Setup details:
Yes, all are L2 transactions. We submit everything to the "main" node via the HTTP API protocol.
hydra version: 1.0.0
Issues we're observing:
Heap overflow: triggered during state replay
Memory leak: during transaction processing

Infrastructure:
4x Hydra nodes + 1 Cardano node
Running in the same Kubernetes cluster on GCP
Happy to share the YAML files if that helps!

Resource allocation:
Host machine: C4d-highcpu-32 (32 vCPUs, 60GB RAM)
Each Hydra node: 4 vCPUs, 7GB RAM
Cardano node: 8 vCPUs, 28GB RAM

What I did was alter existing benchmarks to spin three hydra-node's and then two of them spam the third one with bunch of transactions every 0.01 second.
I tried with both valid and invalid transactions but it didn't make any difference.
What I notice is that the memory usage is increasing very slowly but is still in MBs after couple of minutes of this heavy workflow.
In general I failed to reproduce this issue by altering benchmarks and asked for detailed description on what DD team is doing to stress test hydra nodes.
I will try to reproduce loading of a state file and heap overflow error they are seeing but from what I saw so far I would not expect any special findings since we do stream from disk but perhaps replaying the state could be problematic.

2025-11-04

SB on Partial Fanout

Seems like we are confused regarding the plan on implementing partial fanouts.
I see the approach as follows:
- Alter the SignableRepresentation of a snapshot to include the accumulator and the CRS
- Accumulator must use the utxoHash + any utxo to commit/decommit so it works in fanout (we don't check signatures on fanout but we need this for close and any other potential transactions we want to partially do)
- Now users sign something we can check on-chain
- Calculate proof off-chain and put it in the close datum
- subset of elements can be calculated on-chain by hashing the outputs.
This should give us a way to check the membership of utxo we want to take out of the fanout tx.
We should make the fanout validator generic enough so it can validate all partial fanout transactions until the Head doesn't have any more UTxO to fanout. I think this is ok and not that hard since we rely on redeemer to give us more information on the number of outputs we expect from the fanout tx.
Question is how to organize the off-chain code to do the partial fanout without interacting with the user. We would need to have a way of observing the fanout, then detect that we didn't fanout all of the UTxO and repeat the process. It should also be possible to resume the fanout in case your internal wallet doesn't have any more funds.
Another approach would be to create a batch of fanout transactions and chain and submit them in one go (this sounds like easier thing to do since we don't need to observe and continue)
If we batch transactions then the Final state should come after we observed all partial fanout transactions.
Another thing to think about is how do you actually know that the UTxO set to fanout is too big . We would need to evaluate the transaction and detect if it would fail and reduce the UTxO until we get a valid tx.
Our job right now is to make fanout work by using the checkMembership function and fanout all what we have together. Let's start small.
Another thing to consider is where to put the proof generation? Likely the place is somewhere in the Handler.hs since we have access to IO there.

October 2025

2025-10-06

Some notes on fanout limit

@jmagan observed we can just run our own tx-cost benchmarks:
```
cabal repl hydra-node:bench:tx-cost
> import TxCost
> generate computeFanOutCost
```
If you set the numberOfParties = 1 you get exactly the limit found by @vrom911 !

2025-10-01

SB on deposit reference script

Looking at all this work I realize I made a mess when implementing partial tokens feature. Blueprint tx needs to be the source of truth and we should not add any other inputs/outputs since all the deposit should do can be summarized in the provided blueprint transaction.
This approach should work out for partial tokens as well so this was my oversight and I should revert and delete all code related to tokens/ada splitting and stuff like that since all of this can be achieved using blueprint tx and balancing the transaction in our wallet.
We do need to set the output ourselves so this was probably why we can't just deal with partial tokens easily.
I'll try to see how it would look if we allow the users to specify the output too.
I got to delete a huge chunk of code related to handling different assets and ada for partial deposits Yaaay!
Seems like the idea of using the blueprint outputs for this was not so bad after all.
Doing some changes in how we observe deposits since now we don't depend on blueprint inputs anymore. Is it enough to make sure deposited value is contained in the deposit tx output value?
After consulting with SN the best check would be to make sure the deposit output is completely present in our datum - or check the value since the TxIn's would differ. This is something I'll do tomorrow.
Right now our tests using reference inputs are green! In the process we gained much flexibility so I am very pleased.
Next steps are to make sure all tests are working with the new blueprint logic we have and double check that all funds stay the same after the fanout (using also partial deposits)

September 2025

2025-09-30

SB on deposit reference script

Continuing from yesterday I realize it is not enough to create a reference script UTxO, we also need to have a UTxO at the referenced script address that we want to spend.
While this is true I am still getting MissingScript error and I've exhausted all my options so it is time to collect all information and ask for another set of eyes.
Problem statement:
I want to create script output at unspendable script address (just like we do with publish hydra scripts) and then reference this input to prove that you can deposit using referenced script UTxO.
I first create the output for our dummyValidatorScript present at address of examplePlutusScriptThatAlwaysFails. This is how the tx looks like:

  "f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24"

== INPUTS (1)
- fbad317afbdf139d7029d1f0c52f6a636da66c87b535a32e96ec7a1b75257c73#0
      ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "69830961c6af9095b0f2648dff31fa9545d8f0b6623db865eb78fde8"})) StakeRefNull
      20000000 lovelace
      TxOutDatumNone
      ReferenceScriptNone

== COLLATERAL INPUTS (0)

== REFERENCE INPUTS (0)

== OUTPUTS (2)
Total number of assets: 1
- ShelleyAddress Testnet (ScriptHashObj (ScriptHash "6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a")) StakeRefNull
      14024740 lovelace
      TxOutDatumNone
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "69830961c6af9095b0f2648dff31fa9545d8f0b6623db865eb78fde8"})) StakeRefNull
      5668739 lovelace
      TxOutDatumNone

== TOTAL COLLATERAL
TxTotalCollateralNone

== RETURN COLLATERAL
TxReturnCollateralNone

== FEE
TxFeeExplicit ShelleyBasedEraConway (Coin 306521)

== VALIDITY
TxValidityNoLowerBound
TxValidityUpperBound ShelleyBasedEraConway Nothing

== MINT/BURN
0 lovelace

== SCRIPTS (0)
Total size (bytes):  0

== DATUMS (0)

== REDEEMERS (0)

== REQUIRED SIGNERS
[]

== METADATA
TxMetadataNone

Here you can see I locked my dummyValidatorScript output at script address 6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a which is a hash of examplePlutusScriptThatAlwaysFails
Then I create output at the dummyValidatorScript address that looks like this:

    [
        ( TxIn "42f36eeca3758edc7a28cb512d7979bf0297f45dcf9d0ea96ce9af84cd21c0a8"
            ( TxIx 0 )
        , TxOut
            ( AddressInEra ( ShelleyAddressInEra ShelleyBasedEraConway )
                ( ShelleyAddress Testnet
                    ( ScriptHashObj
                        ( ScriptHash "f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28" )
                    ) StakeRefNull
                )
            )
            ( TxOutValueShelleyBased ShelleyBasedEraConway
                ( MaryValue
                    ( Coin 3000000 )
                    ( MultiAsset
                        ( fromList [] )
                    )
                )
            )
            ( TxOutDatumHash AlonzoEraOnwardsConway "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec" ) ReferenceScriptNone
        )
    ]

Here you can see that we have the output at correct address f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28 and there is a datum hash present (for () datum)
For the reference (no pun intended) here is the referenced UTxO:

      [
        ( TxIn "f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24"
            ( TxIx 0 )
        , TxOut
            ( AddressInEra ( ShelleyAddressInEra ShelleyBasedEraConway )
                ( ShelleyAddress Testnet
                    ( ScriptHashObj
                        ( ScriptHash "6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a" )
                    ) StakeRefNull
                )
            )
            ( TxOutValueShelleyBased ShelleyBasedEraConway
                ( MaryValue
                    ( Coin 14024740 )
                    ( MultiAsset
                        ( fromList [] )
                    )
                )
            ) TxOutDatumNone
            ( ReferenceScript BabbageEraOnwardsConway
                ( ScriptInAnyLang ( PlutusScriptLanguage PlutusScriptV3 )
                    ( PlutusScript PlutusScriptV3
                        ( PlutusScriptSerialised "...redacted..." )
                    )
                )
            )
        )
    ]

Then I want to construct a blueprint transaction to send to Hydra api together with the script utxo. This tx looks like this:

"0ac45566a10b82d6d72f460c40304fedfe55d9c9df6566767e59d5c0e3752418"

== INPUTS (1)
- 42f36eeca3758edc7a28cb512d7979bf0297f45dcf9d0ea96ce9af84cd21c0a8#0
      ShelleyAddress Testnet (ScriptHashObj (ScriptHash "f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28")) StakeRefNull
      3000000 lovelace
      TxOutDatumHash "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
      ReferenceScriptNone

== COLLATERAL INPUTS (0)

== REFERENCE INPUTS (1)
- f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24#0

== OUTPUTS (0)
Total number of assets: 0

== TOTAL COLLATERAL
TxTotalCollateralNone

== RETURN COLLATERAL
TxReturnCollateralNone

== FEE
TxFeeExplicit ShelleyBasedEraConway (Coin 0)

== VALIDITY
TxValidityNoLowerBound
TxValidityUpperBound ShelleyBasedEraConway Nothing

== MINT/BURN
0 lovelace

== SCRIPTS (0)
Total size (bytes):  0

== DATUMS (0)

== REDEEMERS (1)
- ConwaySpending (AsIx {unAsIx = 0}) ( cpu = 0, mem = 0 )
  [0,[]]

== REQUIRED SIGNERS
[]

== METADATA
TxMetadataNone

This finally leads to MissingScript error which I am trying to solve:

       , responseEarlyHints = []}) "ScriptFailedInWallet {redeemerPtr = \"ConwaySpending (AsIx {unAsIx = 0})\", failureReason = \"MissingScript (ConwaySpending (AsIx {unAsIx = 0})) (fromList [(ConwaySpending (AsIx {unAsIx = 0}),(ConwaySpending (AsItem {unAsItem = TxIn (TxId {unTxId = SafeHash \\\"42f36eeca3758edc7a28cb512d7979bf0297f45dcf9d0ea96ce9af84cd21c0a8\\\"}) (TxIx {unTxIx = 0})}),Nothing,ScriptHash \\\"f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28\\\"))])\", failingTx = ShelleyTx ShelleyBasedEraConway (AlonzoTx {body = TxBodyConstr ConwayTxBodyRaw {ctbrSpendInputs = fromList [TxIn (TxId {unTxId = SafeHash \"42f36eeca3758edc7a28cb512d7979bf0297f45dcf9d0ea96ce9af84cd21c0a8\"}) (TxIx {unTxIx = 0})], ctbrCollateralInputs = fromList [], ctbrReferenceInputs = fromList [TxIn (TxId {unTxId = SafeHash \"f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24\"}) (TxIx {unTxIx = 0})], ctbrOutputs = StrictSeq {fromStrict = fromList [Sized {sizedValue = (Addr Testnet (ScriptHashObj (ScriptHash \"ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa"))

After writing this I realize I could provide the reference UTxO in the request also since that might be the problem.
Doing this yields a different error:

       uncaught exception: SubmitTransactionException
       SubmitTxValidationError (TxValidationErrorInCardanoMode (ShelleyTxValidationError ShelleyBasedEraConway (ApplyTxError (ConwayUtxowFailure (UtxoFailure (BabbageNonDisjointRefInputs (TxIn (TxId {unTxId = SafeHash "f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24"}) (TxIx {unTxIx = 0}) :| []))) :| [ConwayUtxowFailure (MissingRequiredDatums (fromList [SafeHash "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"]) (fromList [])),ConwayUtxowFailure (MissingScriptWitnessesUTXOW (fromList [ScriptHash "6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a"]))]))))

This problem has something to do with the UTxO since ledger UTxO is missing the reference input entry!
I created another script instead of alwaysFailing one since I neede Plutus script V3 instead of V1 for setting the aux scripts in blueprint tx setTxAuxScripts.
After this change the deposit tx is balanced but it fails on submission with :

    test/Test/EndToEndSpec.hs:331:7:
  1) Test.EndToEnd, End-to-end on Cardano devnet, single party hydra head, deposit reference script
       uncaught exception: SubmitTransactionException
       SubmitTxValidationError (TxValidationErrorInCardanoMode (ShelleyTxValidationError ShelleyBasedEraConway (ApplyTxError (ConwayUtxowFailure (UtxoFailure (BabbageNonDisjointRefInputs (TxIn (TxId {unTxId = SafeHash "e2a2f0f3bae3b368918471e00fdc1aa9e93af27bc37a3563cbbfc5036ebb6189"}) (TxIx {unTxIx = 0}) :| []))) :| [ConwayUtxowFailure (MissingRequiredDatums (fromList [SafeHash "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"]) (fromList [])),ConwayUtxowFailure (MissingScriptWitnessesUTXOW (fromList [ScriptHash "bc84226deec0c11548c7eccd34701ac717d18ce538033e31652a1ee0"]))]))))

I think the test setup is now fine and deposit fails since I am feeding it a UTxO which contains the reference input I want but the input is duplicated:

"02f4fbbc2c7376374fa648b5aecea55d0e6d5491a2edc95e478d64959de62673"

== INPUTS (3)
- 42f36eeca3758edc7a28cb512d7979bf0297f45dcf9d0ea96ce9af84cd21c0a8#0
      ShelleyAddress Testnet (ScriptHashObj (ScriptHash "f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28")) StakeRefNull
      3000000 lovelace
      TxOutDatumHash "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
      ReferenceScriptNone
- 66ecd46b36d10e6bc21c8640d69da842a6647f65f0900c590cfe9cc47f801e33#0
- dc180ace5d243b793a01dc14a54f6af107fba7257981a97770fe250a3d7a6dd5#1

== COLLATERAL INPUTS (1)
- dc180ace5d243b793a01dc14a54f6af107fba7257981a97770fe250a3d7a6dd5#1

== REFERENCE INPUTS (1)
- 66ecd46b36d10e6bc21c8640d69da842a6647f65f0900c590cfe9cc47f801e33#0

== OUTPUTS (2)
Total number of assets: 1
- ShelleyAddress Testnet (ScriptHashObj (ScriptHash "ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c")) StakeRefNull
      17024740 lovelace
      TxOutDatumInline [0,["0xd0786d92892d904ae16c775e85648c6cb669bd053bfed39c746c06ab",1759246820409,[[0,[[0,["0x42f36eeca3758edc7a28cb512d7979bf0297f45dcf9d0ea96ce9af84cd21c0a8",0]],"0xd8799fd8799fd87a9f581cf4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28ffd87a80ffa140a1401a002dc6c0d87a9f5820923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ecffd87a80ff"]],[0,[[0,["0x66ecd46b36d10e6bc21c8640d69da842a6647f65f0900c590cfe9cc47f801e33",0]],"0xd8799fd8799fd87a9f581c3d552cf2ecfbfff44d3d0fdb469f751846f27a3797c795c79bdd6f2dffd87a80ffa140a1401a00d60024d87980d8799f581cf4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28ffff"]]]]]
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d"})) StakeRefNull
      9506170 lovelace
      TxOutDatumNone

== TOTAL COLLATERAL
TxTotalCollateralNone

== RETURN COLLATERAL
TxReturnCollateralNone

== FEE
TxFeeExplicit ShelleyBasedEraConway (Coin 1771933)

== VALIDITY
TxValidityNoLowerBound
TxValidityUpperBound ShelleyBasedEraConway (Just (SlotNo 24))

== MINT/BURN
0 lovelace

== SCRIPTS (0)
Total size (bytes):  0

== DATUMS (1)
- "923918e403bf43c34b4ef6b48eb2ee04babed17320d8d1b9ff9ad086e86f44ec"
  [0,[]]

== REDEEMERS (1)
- ConwaySpending (AsIx {unAsIx = 0}) ( cpu = 10000000000, mem = 14000000 )
  [0,[]]

== REQUIRED SIGNERS
[]

== METADATA
TxMetadataInEra ShelleyBasedEraConway (TxMetadata {unTxMetadata = fromList [(55555,TxMetaText "HydraV1/DepositTx")]})

We have input 66ecd46b36d10e6bc21c8640d69da842a6647f65f0900c590cfe9cc47f801e33 duplicated since when building a deposit tx we take the whole UTxO and put it in the tx inputs which in this case is wrong.
I need to think about what to do in this situation? How to acquire the needed utxo for the reference input I want to spend? For the head scripts reference inputs we track them in the ScriptRegistry. What to do with the user inputs like this?

2025-09-29

SB on deposit reference script

Current situation - I am trying to spend a reference script into a deposit contract but get:

        , responseEarlyHints = []}) "ScriptFailedInWallet {redeemerPtr = \"ConwaySpending (AsIx {unAsIx = 0})\", failureReason = \"MissingScript (ConwaySpending (AsIx {unAsIx = 0})) (fromList [(ConwaySpending (AsIx {unAsIx = 0}),(ConwaySpending (AsItem {unAsItem = TxIn (TxId {unTxId = SafeHash \\\"f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24\\\"}) (TxIx {unTxIx = 0})}),Nothing,ScriptHash \\\"6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a\\\"))])\", failingTx = ShelleyTx ShelleyBasedEraConway (AlonzoTx {body = TxBodyConstr ConwayTxBodyRaw {ctbrSpendInputs = fromList [TxIn (TxId {unTxId = SafeHash \"f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24\"}) (TxIx {unTxIx = 0})], ctbrCollateralInputs = fromList [], ctbrReferenceInputs = fromList [TxIn (TxId {unTxId = SafeHash \"f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24\"}) (TxIx {unTxIx = 0})], ctbrOutputs = StrictSeq {fromStrict = fromList [Sized {sizedValue = (Addr Testnet (ScriptHashObj (ScriptHash \"ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa"))

This error is weirs since I am actually attaching a script redeemer to the tx so MissingScript is confusing here. Script utxo has the script attached and this is how the bluprint looks like:

  "10f789e7adafb108d289bbd9ac81dd53f2804066d1e136af6235d95e94dfc7f0"

== INPUTS (0)

== COLLATERAL INPUTS (0)

== REFERENCE INPUTS (1)
- f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24#0
      ShelleyAddress Testnet (ScriptHashObj (ScriptHash "6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a")) StakeRefNull
      14024740 lovelace
      TxOutDatumNone
      ReferenceScript PlutusScriptLanguage PlutusScriptV3 "f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28"

== OUTPUTS (0)
Total number of assets: 0

== TOTAL COLLATERAL
TxTotalCollateralNone

== RETURN COLLATERAL
TxReturnCollateralNone

== FEE
TxFeeExplicit ShelleyBasedEraConway (Coin 0)

== VALIDITY
TxValidityNoLowerBound
TxValidityUpperBound ShelleyBasedEraConway Nothing

== MINT/BURN
0 lovelace

== SCRIPTS (1)
Total size (bytes):  3045
- Script (ScriptHash "f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28")

== DATUMS (0)

== REDEEMERS (1)
- ConwaySpending (AsIx {unAsIx = 0}) ( cpu = 0, mem = 0 )
  42

== REQUIRED SIGNERS
[]

== METADATA
TxMetadataNone

What I notice is that reported script hash that is missing is actually the same as the hash of the input not the script itself f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24 so this is probably something to look at to resolve this error.
This was part of the issue reported by the user but what they saw was RedeemerPointsToUnknownScriptHash error.
If I remove the added script witness I get the same error:

         , responseEarlyHints = []}) "ScriptFailedInWallet {redeemerPtr = \"ConwaySpending (AsIx {unAsIx = 0})\", failureReason = \"MissingScript (ConwaySpending (AsIx {unAsIx = 0})) (fromList [(ConwaySpending (AsIx {unAsIx = 0}),(ConwaySpending (AsItem {unAsItem = TxIn (TxId {unTxId = SafeHash \\\"f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24\\\"}) (TxIx {unTxIx = 0})}),Nothing,ScriptHash \\\"6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a\\\"))])\", failingTx = ShelleyTx ShelleyBasedEraConway (AlonzoTx {body = TxBodyConstr ConwayTxBodyRaw {ctbrSpendInputs = fromList [TxIn (TxId {unTxId = SafeHash \"f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24\"}) (TxIx {unTxIx = 0})], ctbrCollateralInputs = fromList [], ctbrReferenceInputs = fromList [TxIn (TxId {unTxId = SafeHash \"f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24\"}) (TxIx {unTxIx = 0})], ctbrOutputs = StrictSeq {fromStrict = fromList [Sized {sizedValue = (Addr Testnet (ScriptHashObj (ScriptHash \"ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa"))

After thinking about this I think I need to separate publishing of the script I want to use as a reference so it happens before the hydra-node starts. I think hydra-node is not aware of this script since it is not needed anywhere in Hydra protocol (we don't observe it) and if just publish before starting hydra-node then it would get picked up.
What is happening here? I see the UTxO with the correct script hash right before we try to do script evaluation but I still get MissingScript error?
Ok, maybe I am mistaking. I'll dump all data there is to see. I expect the dummyValidatorScript to hash to f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28
Actually I am not crazy. This is the utxo entry right before evaluating scripts:

        ,
           ( TxIn
               ( TxId
                   { unTxId = SafeHash "f15c9e1a6b4ab72c112b4f9b06ba9c38de9ced660a3c836f93b43b27222c1c24" }
               )
               ( TxIx
                   { unTxIx = 0 }
               )
           ,
               ( Addr Testnet
                   ( ScriptHashObj
                       ( ScriptHash "6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a" )
                   ) StakeRefNull
               , MaryValue
                   ( Coin 14024740 )
                   ( MultiAsset
                       ( fromList [] )
                   )
               , NoDatum
               , SJust PlutusScript PlutusV3 ScriptHash "f4089a5295438315b557fb295de27eeb962b0a412ce48bb8be070a28"
               )
           )

So I think the script is attached correctly but there could be problems with using the wrong datum/redeemer. Right now I am using a dummy datum (the script should accept any) so I'll check this next.
We do use the same plutus versions so this should not be the problem. Also the always failing script hashes to 6a09cb22defaf4a96a6be1ef6c07467ac9923d1750a79214a06c503a so the UTxO address is correct.
The only thing that trips me is that I see NoDatum in the UTxO entry but I do set the datum in the script redeemer and also there is a script present in the UTxO.

2025-09-24

SB on Best practise on committing user scripts

So now I want to write out a test which uses the script I created yesterday.
At first, I thought it would be hard to maintain this test but I realized we have a way of obtaining the script hashes through hydraScriptCatalogue
Setting the redeemer and using the exampleSecureValidatorScript yields:

, responseEarlyHints = []}) "ScriptFailedInWallet {redeemerPtr = \"ConwaySpending (AsIx {unAsIx = 1})\", failureReason = \"MissingScript (ConwaySpending (AsIx {unAsIx = 1})) (fromList [(ConwaySpending (AsIx {unAsIx = 0}),(ConwaySpending (AsItem {unAsItem = TxIn (TxId {unTxId = SafeHash \\\"44d2f08229825ba3a3e260ecd2dcc9ac0af4da6462bcf5f435d0b0ccca90dbce\\\"}) (TxIx {unTxIx = 1})}),Just (ConwayPlutusV3 (Plutus {plutusBinary = \\\"WQpZAQEAMyMjIyMjIyMjIyIyJTMwBTIyMjIyUzIzALMAEwDTdUAEJkZkSmZmZgLADCZGRkZGSmZgJmAIACKmZgLmAsbqgCwAgEhUzMBMwCQARMlMzAYABATEyUzMzMB0AEBQBQTJTMwGjAdADEzAIABIlMzAcACAHEyUzMzMCEAETMwCQARMAIwIAAwGAGAGAGAGDAeACAVN1gAICgChgNAAmAsbqgCwEjAUN1QBQqZmAiYARgJm6oAQTIyMjIyMjJTMwGDAJMBo3VAICZgCJIEDSTAxADNw5mYAJurMAYwGzdUAiAOkQQtIeWRyYUhlYWRWMQBIAETIyMjIyMjIyMjIyMjIyMjIyMjIyUzMzMDQAEAIAITIzAfABIlMzAzACEyMlMzAxMwHUkEDSTEzADNx4EBuuMBQwNDdUAQKmZgYmRkZKZmBoYEoAImBCkkDSTA1ABUzMDQwKgARMjJTMwOgAQAhUzMDowPQARMjMCNJAQNJMDIAMjMAEAEAciUzMD0AEUoCZkSmZgdmbjwAgBRSiJmAIAIACbrjA/ABMEAAE3XGB4ACAEZGYAIAIAhEpmYHYAIpeuATIzMiJTMwOzAxACEzBAN1IAZmAMAMACJmAMAMACbrj"))

I noticed script witness was wrong (since of copy/pasta mistake)
Now we finally see an error coming from our new secure validator Initial input not found:

        , responseEarlyHints = []}) "ScriptFailedInWallet {redeemerPtr = \"ConwaySpending (AsIx {unAsIx = 1})\", failureReason = \"ValidationFailure (WrapExUnits {unWrapExUnits = ExUnits' {exUnitsMem' = 0, exUnitsSteps' = 0}}) (CekError An error has occurred:\\nThe machine terminated because of an error, either from a built-in function or from an explicit use of 'error'.) [\\\"Initial input not found\\\",\\\"PT5\\\"] (PlutusWithContext {pwcProtocolVersion = Version 9, pwcScript = Left (Plutus {plutusBinary = \\\"WRMyAQEAMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMikyMjJVMzVzRm4dIAAAIRMjIygAmRkZKpmauaM3DpAAABCJkZGRkZGRkZGRkZGRkZGRlABMjUBw3WAAmroQEZkagOm6wAE1dCAhMjUCA3WAAmroQD5utNXQgHTMAEwAXWmroQDZkagRG6wAE1dCAZMzAnAldaauhALmRkZKpmauaM3DpAAABCJkZQATIyMlUzNXNGbh0gAAAhGACZgCutNXQgAzAENXQmrogAQiYGqSAQNQVDEANVc8AEaq50AE3VGroQA5kZGSqZmrmjNw6QAAAQjABMwBXWmroQAZgCGroTV0QAIRMDVJAQNQVDEANVc8AEaq50AE3VGroTV0QAYjIyMlUzNXNGbh0gAAAhGAEIqpmauaM3DpABABCMAARMDVJAEDUFQxADVXPABGqudABN1QAJEZGRkqmZq5ozcOkAAAEIwAhFVMzVzRm4dIAIAIRgAmAKauhABCKqZmrmjNw6QAgAQjAEETA1SRA1BUMQA1VzwARqrnQATdUACImBgkg"))

After this I just made sure I see the expected errors each time I use wrong headId or initial script hash.

2025-09-23

SB on Best practise on committing user scripts

We need to help our users do the checks that would make sure their validators are checking that commit goes into the right Head.
In this process we also want to build some documentation around committing scripts and check if we are able to commit different script UTxO (with inline datums, using datum hash etc.)
Looking at existing guides on script UTxO - I think I should just alter this tutorial/how-to to include the parts about the validator checks.
Actually, since I would want to also create a test as the outcome of this (primarily) documentation task perhaps it is a bit better to have a dedicated tutorial about this and talk about the Head security there.
Writing things out seems like such good exercise! I haven't really done plutus in a long time so things are interesing.
Question arises: How to calculate the script address needed to check on-chain if there is a initial input that is spent? One can use cardano-addresses for that so I added it to our flake.
Actually I didn't need to use address, script hash is enough since this is also the address of a script in Plutus.
I created a simple validator and added some nice documentation explaining stuff along the way.
The next step is to actually try to use the script and commit it to the head so it leaves traces in public explorers. If I decide to write a test for it, since we need to use correct initial script hash, it would become obsolete sooner or later when the scripts change and we would need to maintain it (or generalize enough).
I think having some example that could be verified and looked up on public explorers should be enough for this demonstration.

2025-09-22

SB on Blockfrost smoke tests

Our nightly CI tests fail when trying to run end-to-end tests using BF backend.
Locally I see the same behaviour: Seems like the fee calculation is off:

   test/Test/EndToEndSpec.hs:255:7:
  1) Test.EndToEnd, End-to-end on Cardano devnet, single party hydra head, full head life-cycle
       uncaught exception: BlockfrostException
       BlockfrostAPIError "BlockfrostBadRequest \"{\\\"contents\\\":{\\\"contents\\\":{\\\"contents\\\":{\\\"era\\\":\\\"ShelleyBasedEraConway\\\",\\\"error\\\":[\\\"ConwayUtxowFailure (UtxoFailure (FeeTooSmallUTxO (Mismatch {mismatchSupplied = Coin 207653, mismatchExpected = Coin 227805})))\\\"],\\\"kind\\\":\\\"ShelleyTxValidationError\\\"},\\\"tag\\\":\\\"TxValidationErrorInCardanoMode\\\"},\\\"tag\\\":\\\"TxCmdTxSubmitValidationError\\\"},\\\"tag\\\":\\\"TxSubmitFail\\\"}\""

There is only couple of lovelaces difference so I wonder if this has to do with the cardano-node version? Perhaps blockfrost uses different cardano-node version which gives us back different pparams yielding different/wrong fees? This doesn't make too much sense to me but it was the first thing that popped into my mind.
The failure happens when we want to refuelIfNeeded Seems like makeTransactionBodyAutoBalance is not calculating fees in case of BF backend?
Perhaps there was an exception and then retryOnExceptions is firing too soon for BF integration (1 second). If I bump it further perhaps we see some green tests.
I've printed the failing tx:

  "063fc1a2a8681f74cbe9b60e5181c11032ef0cf6787ca4ca1bffcf55c6da29d7"

== INPUTS (1)
- b1045e7172a4055865f4774cf2662f4004fad84ced3d708445dad03233e4bfd9#1
      ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "9783be7d3c54f11377966dfabc9284cd6c32fca1cd42ef0a4f1cc45b"})) StakeRefNull
      109265874052 lovelace
      1 13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3.487964726120446f6f6d202d2033726420506c6163652054726f706879
      1 19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07.487964726120446f6f6d202d20326e6420506c6163652054726f706879
      1 1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11.2331
      1 6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a.2331
      1 cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b.2331
      69918000000 d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4.0014df105553444d
      1 dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee.487964726120446f6f6d202d2034746820506c6163652054726f706879
      TxOutDatumNone
      ReferenceScriptNone

== COLLATERAL INPUTS (1)
- b1045e7172a4055865f4774cf2662f4004fad84ced3d708445dad03233e4bfd9#1
      ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "9783be7d3c54f11377966dfabc9284cd6c32fca1cd42ef0a4f1cc45b"})) StakeRefNull
      109265874052 lovelace
      1 13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3.487964726120446f6f6d202d2033726420506c6163652054726f706879
      1 19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07.487964726120446f6f6d202d20326e6420506c6163652054726f706879
      1 1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11.2331
      1 6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a.2331
      1 cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b.2331
      69918000000 d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4.0014df105553444d
      1 dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee.487964726120446f6f6d202d2034746820506c6163652054726f706879
      TxOutDatumNone
      ReferenceScriptNone

== REFERENCE INPUTS (0)

== OUTPUTS (2)
Total number of assets: 13
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d"})) StakeRefNull
      55000000 lovelace
      TxOutDatumNone
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "9783be7d3c54f11377966dfabc9284cd6c32fca1cd42ef0a4f1cc45b"})) StakeRefNull
      109210666399 lovelace
      1 13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3.487964726120446f6f6d202d2033726420506c6163652054726f706879
      1 19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07.487964726120446f6f6d202d20326e6420506c6163652054726f706879
      1 1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11.2331
      1 6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a.2331
      1 cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b.2331
      69918000000 d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4.0014df105553444d
      1 dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df.487964726120446f6f6d202d2031737420506c6163652054726f706879
      1 fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee.487964726120446f6f6d202d2034746820506c6163652054726f706879
      TxOutDatumNone

== TOTAL COLLATERAL
TxTotalCollateral BabbageEraOnwardsConway (Coin 311480)

== RETURN COLLATERAL
TxReturnCollateral BabbageEraOnwardsConway (TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "9783be7d3c54f11377966dfabc9284cd6c32fca1cd42ef0a4f1cc45b"})) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 109265562572) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3"},fromList [("487964726120446f6f6d202d2033726420506c6163652054726f706879",1)]),(PolicyID {policyID = ScriptHash "19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07"},fromList [("487964726120446f6f6d202d20326e6420506c6163652054726f706879",1)]),(PolicyID {policyID = ScriptHash "1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11"},fromList [("2331",1)]),(PolicyID {policyID = ScriptHash "6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828"},fromList [("487964726120446f6f6d202d2031737420506c6163652054726f706879",1)]),(PolicyID {policyID = ScriptHash "ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9"},fromList [("487964726120446f6f6d202d2031737420506c6163652054726f706879",1)]),(PolicyID {policyID = ScriptHash "bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c"},fromList [("487964726120446f6f6d202d2031737420506c6163652054726f706879",1)]),(PolicyID {policyID = ScriptHash "c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a"},fromList [("2331",1)]),(PolicyID {policyID = ScriptHash "cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e"},fromList [("487964726120446f6f6d202d2031737420506c6163652054726f706879",1)]),(PolicyID {policyID = ScriptHash "d0c91707d75011026193c0fce742443dde66fa790
936981ece5d9f8b"},fromList [("2331",1)]),(PolicyID {policyID = ScriptHash "d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4"},fromList [("0014df105553444d",69918000000)]),(PolicyID {policyID = ScriptHash "dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df"},fromList [("487964726120446f6f6d202d2031737420506c6163652054726f706879",1)]),(PolicyID {policyID = ScriptHash "fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee"},fromList [("487964726120446f6f6d202d2034746820506c6163652054726f706879",1)])])))) TxOutDatumNone ReferenceScriptNone)

== FEE
TxFeeExplicit ShelleyBasedEraConway (Coin 207653)

== VALIDITY
TxValidityNoLowerBound
TxValidityUpperBound ShelleyBasedEraConway Nothing

== MINT/BURN
0 lovelace

== SCRIPTS (0)
Total size (bytes):  0

== DATUMS (0)

== REDEEMERS (0)

== REQUIRED SIGNERS
[]

== METADATA
TxMetadataNone

Which yields this error:

  test/Test/EndToEndSpec.hs:255:7:
  1) Test.EndToEnd, End-to-end on Cardano devnet, single party hydra head, full head life-cycle
       uncaught exception: BlockfrostException
       BlockfrostAPIError "BlockfrostBadRequest \"{\\\"contents\\\":{\\\"contents\\\":{\\\"contents\\\":{\\\"era\\\":\\\"ShelleyBasedEraConway\\\",\\\"error\\\":[\\\"ConwayUtxowFailure (UtxoFailure (FeeTooSmallUTxO (Mismatch {mismatchSupplied = Coin 207653, mismatchExpected = Coin 227805})))\\\"],\\\"kind\\\":\\\"ShelleyTxValidationError\\\"},\\\"tag\\\":\\\"TxValidationErrorInCardanoMode\\\"},\\\"tag\\\":\\\"TxCmdTxSubmitValidationError\\\"},\\\"tag\\\":\\\"TxSubmitFail\\\"}\""

This tells us that we calculated fee to be 207653 while BF rejected this tx since it expected a bit higher fee 227805. So the difference in lovelace is 20152.
I need to also check pparams between BF and cardano-node runs since they play major part in how the tx end up looking.
I've printed the tx and pparams using direct backend and what I notice is that BF backend gives back a different UTxO for faucet. BF seems to return existing assets together with ADA and I think this is what is causing the balance errors we see.
Ok so it seems BF is at Protocol version 10 while locally we are at 9 with direct backend. I will put this issue as blocked until we align with the same protocol version. Things should just start working after that.

2025-09-16

SB on H4 in Increment tx

When working on partial commits I noticed something weird - occasionally you would get H4 validator error from evaluating increment tx and what is weird I noticed it has something to do with the amount of generated tokens for the deposit.
If I just seed more ada then the test is always green, but if I use 2 or 3 ada to deposit together with some tokens then I experience H4 or HeadValueIsNotPreserved error.
I understand that if we are trying to transfer large quantities of tokens can have impact on the tx fee but why would we get the validator error beats me!
What I can do is try to debug and figure out what is causing this - I have a pretty good idea where this happens (Wallet.hs) and when we try to estimateScriptsCost for some reason we get ErrScriptExecutionFailed which is then re-mapped to ScriptExecutionFailure in Direct/Handlers.hs.
This means that evalTxExUnits is also running the Head script and checkIncrement validator function.
It is worth mentioning that we either get H4 or :

    1) Test.EndToEnd, End-to-end on Cardano devnet, single party hydra head, can deposit partial UTxO
       uncaught exception: FaucetException
       FaucetFailedToBuildTx {reason = TxBodyErrorMinUTxONotMet (TxOutInAnyEra ConwayEra (TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "d99b3e7c965d67b5e182e93ca8723fc3b61eeb5d9276cd3c2e461a5e"})) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 2000000) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75"},fromList [("006cc09c1b4e884bd6265c5604c5c8b0a162ee09b0c7cf4bde379f9aebec",1),("35bf4a530304089dbac2fed26fccedf510327acf9ebb25",7889008373497912565),("36",2),("37",8130754424926331177),("38",6083698601569809568),("48d44378c0cff4ba1a13e3f6eaa0d738e335d905550aacca67f70688",2),("4bbdd3d3264e6f85c3a65b819a32bc056942b43d8c0f",1319783183065313421),("71a21566f4f89c51730021c4b21d1ab3c35b169d22ca",8985411846134803463),("92261938b47e873dc018600f5f3dadfb5c20754b679c9810d4be81099ac0",8588183237463905639),("9481e44d64abd5d5471d84b12842",7629825169502544102),("a13aa4ac445c248686e3b3594263f770d45ced6caa826499cb265c917450f933",2),("a3e4e135016a6ddc59d2655001a7c044eca612272aefa5b2c1",404060153465380944),("a6e62d57edb79f60",3271098189746201174),("beba62802da1bd771798cc138337360927608e7901b0",5896190021190470668),("cf3a6e",2),("e4a201af38419bf51bb1c37fb30a3c",2)])])))) TxOutDatumNone ReferenceScriptNone)) (Coin 2689440)}

  To rerun use: --match "/Test.EndToEnd/End-to-end on Cardano devnet/single party hydra head/can deposit partial UTxO/" --seed 364461330

So sometimes we get failures when trying to mint tokens from faucet too.
There is also evalTxExUnitsWithLogs which provides logs useful for debugging!
I sprinkled some logging and used evalTxExUnitsWithLogs (not sure how useful it is to print only when we get Right result) using the seed:

  cabal test hydra-cluster --test-options '--match "can deposit partial UTxO" --seed 185090575'

and what is weird is this seed I got from the first unsuccessful run (H4) but running for the second time I see:

  test/Test/EndToEndSpec.hs:330:7:
  1) Test.EndToEnd, End-to-end on Cardano devnet, single party hydra head, can deposit partial UTxO
       uncaught exception: FaucetException
       FaucetFailedToBuildTx {reason = TxBodyErrorMinUTxONotMet (TxOutInAnyEra ConwayEra (TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "fecd3d8bda4e7f450205cf5c2756d39b79aa81378f4bc9e8171cfcd8"})) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 2000000) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75"},fromList [("2676d60be23ff11bad4666202ad03aefab9316",1),("2a17115569c616af8053",2),("2cabddce782763098b1b39901e0d39a94f0915e669",1),("314b8161d03d3638",1),("32",6886952548160811876),("33",4498128670021801312),("36",1),("36c74fd7aadc801a595f66286b8c98c7899ec7d06ec269f7",2),("39",13812362742214938515),("44e59ea5093c3403b4373b2e3c52112c6231f6",2),("653c9f0794b0",7611254910991885205),("86f0d82b5d570b8ed0139015be250878b34ca5c9934b08",7709398943983541368),("876ab1d4d53f9bea81a90e1ffcf4a6a56c",273705404968363380),("d2063b5e504dd4ae1c1fc64e89c67220f9cdc94f2297977dd3864fdf6d32",2),("e1252e9d6a1fb84b32e9f5204818d321a44a",487284472556764299)])])))) TxOutDatumNone ReferenceScriptNone)) (Coin 2228270)}

So the second time faucet failed to build minting tx because of TxBodyErrorMinUTxONotMet
When printing increment and deposit txs I noticed that increment is using always the first deposit output while in this test we have two outputs since we are giving back the change containing leftover assets to the user! This should be the cause of all problems in case these outputs in reverse order!
This is the deposit UTxO we tried to consume in the increment:

  DEPOSIT UTxO: fromList
    [
        ( TxIn "39eda4cd0fd667dd311d2266d022ff98770962e264b89c1a76a92f509b829e30"
            ( TxIx 0 )
        , TxOut
            ( AddressInEra ( ShelleyAddressInEra ShelleyBasedEraConway )
                ( ShelleyAddress Testnet
                    ( ScriptHashObj
                        ( ScriptHash "ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c" )
                    ) StakeRefNull
                )
            )
            ( TxOutValueShelleyBased ShelleyBasedEraConway
                ( MaryValue
                    ( Coin 2935110 )
                    ( MultiAsset
                        ( fromList
                            [
                                ( PolicyID
                                    { policyID = ScriptHash "dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75" }
                                , fromList
                                    [
                                        ( "35"
                                        , 1832708093535842422
                                        )
                                    ,
                                        ( "36"
                                        , 5955814720656890312
                                        )
                                    ,
                                        ( "73dc5dc233d70942c576a06c06c2a727adbbfa5b"
                                        , 6167589719126987963
                                        )
                                    ,
                                        ( "e47cfedc29b00ab35ac2da15be40b2ad184d39e01dd0bee65a2eb5c0"
                                        , 3397913103788772526
                                        )
                                    ,
                                        ( "ee1483eef61368f5a22f341e18c3a12bcdd4f5b61a33ca0c"
                                        , 8511226188589623859
                                        )
                                    ]
                                )
                            ]
                        )
                    )
                )
            )

This is how deposit tx looks like:

"39eda4cd0fd667dd311d2266d022ff98770962e264b89c1a76a92f509b829e30"

== INPUTS (2)
- 04e36b0eeb5d97be19e543583e3b736640b45ceaf5ece1d37f5db9ac88143f20#1
- bdf0688131efb9c476553b5d60d1a52f4814e11cc9359f3f19f0345619683f12#0
      ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "884edab29cc9791346a0d93881c678d136542dade7929132b7b315da"})) StakeRefNull
      10000000 lovelace
      2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.03dd2e426d18a75446
      2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.34
      1832708093535842422 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.35
      5955814720656890312 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.36
      2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.37
      6167589719126987963 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.73dc5dc233d70942c576a06c06c2a727adbbfa5b
      4 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.74ad1fc65024c6
      3 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.823b659d6d43680c260d0109c4712a5f767b82
      1 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.a8165b9ada218c52340576a3caaba0ca70c272ba
      2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.c57dfc865840dccc5feff758db02c09e2d9ed34f7035154ce8
      3397913103788772526 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.e47cfedc29b00ab35ac2da15be40b2ad184d39e01dd0bee65a2eb5c0
      1 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.e54e1616e3abbd03bfb2
      8511226188589623859 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.ee1483eef61368f5a22f341e18c3a12bcdd4f5b61a33ca0c
      4 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.f85c0b51cb3a3f67a227b9b89d6d985dbb5f25787230
      TxOutDatumNone
      ReferenceScriptNone

== COLLATERAL INPUTS (1)
- 04e36b0eeb5d97be19e543583e3b736640b45ceaf5ece1d37f5db9ac88143f20#1

== REFERENCE INPUTS (0)

== OUTPUTS (3)
Total number of assets: 15
- ShelleyAddress Testnet (ScriptHashObj (ScriptHash "ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c")) StakeRefNull
      2935110 lovelace
      1832708093535842422 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.35
      5955814720656890312 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.36
      6167589719126987963 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.73dc5dc233d70942c576a06c06c2a727adbbfa5b
      3397913103788772526 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.e47cfedc29b00ab35ac2da15be40b2ad184d39e01dd0bee65a2eb5c0
      8511226188589623859 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.ee1483eef61368f5a22f341e18c3a12bcdd4f5b61a33ca0c
      TxOutDatumInline [0,["0xe9e21b6e4f59b3c62721c99109e76abb46e018e4fc184b16e58d650a",1758033905909,[[0,[[0,["0xbdf0688131efb9c476553b5d60d1a52f4814e11cc9359f3f19f0345619683f12",0]],"0xd8799fd8799fd8799f581c884edab29cc9791346a0d93881c678d136542dade7929132b7b315daffd87a80ffa240a1401a0016e360581cdcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75a541351b196f164d0c2ff07641361b52a74deee4cbd9c85473dc5dc233d70942c576a06c06c2a727adbbfa5b1b5597ae2c18ea54bb581ce47cfedc29b00ab35ac2da15be40b2ad184d39e01dd0bee65a2eb5c01b2f27cff708f98cae5818ee1483eef61368f5a22f341e18c3a12bcdd4f5b61a33ca0c1b761df31fc5e2ea33d87980d87a80ff"]]]]]
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "884edab29cc9791346a0d93881c678d136542dade7929132b7b315da"})) StakeRefNull
      8500000 lovelace
      2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.03dd2e426d18a75446
      2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.34
      2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.37
      4 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.74ad1fc65024c6
      3 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.823b659d6d43680c260d0109c4712a5f767b82
      1 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.a8165b9ada218c52340576a3caaba0ca70c272ba
      2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.c57dfc865840dccc5feff758db02c09e2d9ed34f7035154ce8
      1 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.e54e1616e3abbd03bfb2
      4 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.f85c0b51cb3a3f67a227b9b89d6d985dbb5f25787230
      TxOutDatumNone
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d"})) StakeRefNull
      20141053 lovelace
      TxOutDatumNone

== TOTAL COLLATERAL
TxTotalCollateralNone

== RETURN COLLATERAL
TxReturnCollateralNone

== FEE
TxFeeExplicit ShelleyBasedEraConway (Coin 210953)

== VALIDITY
TxValidityNoLowerBound
TxValidityUpperBound ShelleyBasedEraConway (Just (SlotNo 39))

== MINT/BURN
0 lovelace

== SCRIPTS (0)
Total size (bytes):  0

== DATUMS (0)

== REDEEMERS (0)

== REQUIRED SIGNERS
[]

== METADATA
TxMetadataInEra ShelleyBasedEraConway (TxMetadata {unTxMetadata = fromList [(55555,TxMetaText "HydraV1/DepositTx")]})

Hmm so far deposit UTxO is picked correctly... I am debugging wallet code since there I should find the problem.
Nothing suspicious wherever I look.
What I could do is to increase the seeded ADA which will make the test pass and then look at the same print debugging to try and spot where is the difference.
~~I commented out completely mustIncreaseValue which was a failing check and then the test fails on step where we want to see HeadIsFinalized message.~~ This was just a fluke, probably something weird that happened once.
It is worthwhile mentioning I occasionally also see:

  test/Test/EndToEndSpec.hs:330:7:
 1) Test.EndToEnd, End-to-end on Cardano devnet, single party hydra head, can deposit partial UTxO
      uncaught exception: Error
      BearerClosed "<socket: 17> closed when reading data, waiting on next header True"

Ok so I since bumping ada amount solves the issue and I couldn't find anything else problematic then it has to be the ADA amount that is off in the validator check.
Indeed! Deposit output gets autobalanced and it is 2512730 lovelace instead of 150000 which I deliberately specified. This makes the validator check fail and it seems that all is good.
I am glad I did this although it wasn't necessary since everything works - at least now I know the reason and perhaps this should be mentioned somewhere in the documentation since it can be a foot gun.

2025-09-08

SB on partial tokens output

Still trying to figure out what is happening with deposits. What I see is after posting a deposit tx no activity in hydra-nodes a part from PeerConnected messages. It seems like there is an exception raised somewhere (since I also see userInterrupt in stdout) but for the live of me I can't figure out where it is.
Running other deposit related tests to see if they are broken too now.
Ok, other tests still work..phew
Seems like we see a deposit tx arrive from the chain component but there is not subsequent observation and increment posting. Sprinkling some debugging into observation to see why the deposit was not observed.
Ok, I am on to something. We have a check to make sure all deposit inputs are spent into a deposit tx but now with partial deposits this check is not working anymore since we potentially consume some inputs but return to the user a change.
This check doesn't work no more since I am bumping the TxIx when adding tokens to the outputs. Perhaps I should re-think this and merge UTxO contents instead if possible?
In general - is it safe to split complex UTxO like I am doing now? I think this is bound to have problems when users start committing script UTxO with some tokens that also rely on correct datum/redeemer in the UTxO - it is very easy to omit datum/redeemer which can cause problems down the line.
I see H4 when trying to post Increment tx so there are couple of problems with my implementation so far.
Let's try to focus on first one - to not have to bump TxId indices and introduce a function to merge UTxO contents (mappend (<>) just ignores subsequent inputs in case TxIn is already present)
After writing the mergeUTxO function I see H4 on Increment transaction so let's fix this one next.
In the meantime I had to juggle a bit the utxo to deposit since I needed to alter the seeded value and reduce the ADA amount (since we specified partial commit for both ADA and some assets)
I am currently debugging Increment tx so I left debug prints in code. Everywhere I look the values are correct so I need to dig deeper :TM:
BTW seems like ada value in increment is off by 1 ADA? Let's explore tomorrow.

2025-09-03

SB on partial tokens output

Continuing where I left off - it seems like there are still problems with the deposit construction since I am not seeing any tokens at the deposit output. Debugging.
The deposit output with tokens was squashed into the first UTxO containing only ada because they were using the same TxIn. I made sure to bump the index and now I see correctly tokens in the deposit output.
Another problem is splitting what is left and returning to the user. withoutUTxO is not removing the tokens for the same reason - now the TxIn is unique so withoutUTxO is happy to keep the tokens around.
What I did was to not bump indices until the very end. This allows me to withoutUTxO to remove what is to be committed token wise and then create additional leftover output in case we do not want to commit all tokens in the UTxO.
I also notice that we are keeping the lovelace for UTxO with tokens which then produces the wrong outcome - if user specified any lovelace amount to keep this should be respected.
This whole issue feels like it needed grooming before I started to work on it but the problem, I believe, is that we started with only lovelace - now we want to limit tokens as well and this needed a careful approach.
One period of time to get rid of our technical debt would be in order.
It is evan hard to explain in couple of sentences the problems we may have. I think I should perhaps unify functions that:
split the utxo by the specified amount
- split the utxo by the specified assets
This way we would avoid unnecessary juggling with avaliable UTxO
Feels like in general that it would be useful to write a function squashUTxO that will merge all assets under a single TxIn. After having this then we should write another function to operate on this single entry UTxO and take split it into two - what we want to commit and what should be given back to the user.
In the middle of trying things out I realize there will be use cases where people require datums to be present at exact output and similar so it is not that easy as it sounds.
Been running in circles forever! Seems like deposit tx is finally accepted by the node but when waiting on CommitApproved message I am unable to see it.
Ironically after the test times out I do see CommitApproved message in stdout!
Is this a timing issue? Some quirk where I need to pick correct values for deposit/contestation period?

2025-09-03

SB on partial tokens output

It seem there is issue with partial tokens feature where the specified assets are not to be found in the deposit output.
I'll take a look at the Deposit.hs first to glance over it trying to spot the problem.
Immediately I spot this line:

    let leftoverUTxO = (leftoverUTxO' `withoutUTxO` tokensToDepositUTxO)

Here leftoverUTxO' should contain only non ada assets and then we are trying to remove also any tokens we want to commit?
We need a proper e2e test for this.
In order to do this successfully I need to make sure we can mint some assets when requesting funds from faucet.
At first I used an extra argument for the assets that should be minted in the transaction but realized this can be more elegant if I just search for any non ada values and if any is present I use dummyValidatorScript as the script witness for minting.
For some reason I am getting negative values in the tx outputs even though I made sure to filter out the negatives:

  1) Test.EndToEnd, End-to-end on Cardano devnet, single party hydra head, can deposit partial UTxO
       uncaught exception: FaucetException
       FaucetFailedToBuildTx {reason = TxBodyErrorBalanceNegative (Coin 899849338161) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "2e12c5e499e0521b13837391beed1248a2e36117370662ee75918b56"},fromList [("2affd3bbefc1d1b8fd5b684c69f5f2a7f3",-1)]),(PolicyID {policyID = ScriptHash "707aab0cefa84f37cb8ac4c6e7f02de5c3fbda4534b59b53490e433b"},fromList [("33",-2)]),(PolicyID {policyID = ScriptHash "8f461954fe2f18fee1dca233f358907e643ff839ed1f995e4bf325e3"},fromList [("32",-6410724724660021339)])]))}

This happens because I was setting minting values correctly but only passed in UTxO that contains only lovelace.
Had to attach a minting script and ~~optimize it using plutus-tx plugin pragma since the tx got too big (MaxTxSizeUTxO error)~~ make sure all minted tokens use the same script hash for PolicyId but still get:

       uncaught exception: FaucetException
     FaucetFailedToBuildTx {reason = TxBodyScriptExecutionError [(ScriptWitnessIndexMint 0,ScriptErrorMissingScript (ScriptWitnessIndexMint 0) (ResolvablePointers ShelleyBasedEraConway (fromList [(ConwayMinting (AsIx {unAsIx = 0}),(ConwayMinting (AsItem {unAsItem = PolicyID {policyID = ScriptHash "54c25c25088dad1c2421be9b91aa48ca787a18db1d29f38319b8e998"}}),Nothing,ScriptHash "54c25c25088dad1c2421be9b91aa48ca787a18db1d29f38319b8e998")),(ConwayMinting (AsIx {unAsIx = 1}),(ConwayMinting (AsItem {unAsItem = PolicyID {policyID = ScriptHash "8894a9e32b4c206bcc49250d2bcc048466b1dd3dde49ca6fcab8566a"}}),Nothing,ScriptHash "8894a9e32b4c206bcc49250d2bcc048466b1dd3dde49ca6fcab8566a"))]))),(ScriptWitnessIndexMint 1,ScriptErrorMissingScript (ScriptWitnessIndexMint 1) (ResolvablePointers ShelleyBasedEraConway (fromList [(ConwayMinting (AsIx {unAsIx = 0}),(ConwayMinting (AsItem {unAsItem = PolicyID {policyID = ScriptHash "54c25c25088dad1c2421be9b91aa48ca787a18db1d29f38319b8e998"}}),Nothing,ScriptHash "54c25c25088dad1c2421be9b91aa48ca787a18db1d29f38319b8e998")),(ConwayMinting (AsIx {unAsIx = 1}),(ConwayMinting (AsItem {unAsItem = PolicyID {policyID = ScriptHash "8894a9e32b4c206bcc49250d2bcc048466b1dd3dde49ca6fcab8566a"}}),Nothing,ScriptHash "8894a9e32b4c206bcc49250d2bcc048466b1dd3dde49ca6fcab8566a"))])))]}

It seems like there are two script hashes in the minting redeemer. I'll debug to figure it out.
Yes, the code to set the specified policy id was wrong.
After seeing retries when submitting a faucet seed transaction I had to debug again to see the actual error. It was missing collateral since we now have scripts in our plain old faucet tx.

After adding a collateral inputs I finally have a regression test that fails with this error for deposit tx:

  test/Test/EndToEndSpec.hs:324:7:
1) Test.EndToEnd, End-to-end on Cardano devnet, single party hydra head, can deposit partial UTxO
     uncaught exception: SubmitTransactionException
     SubmitTxValidationError (TxValidationErrorInCardanoMode (ShelleyTxValidationError ShelleyBasedEraConway (ApplyTxError (ConwayUtxowFailure (UtxoFailure (ValueNotConservedUTxO (Mismatch {mismatchSupplied = MaryValue (Coin 35583235) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75"},fromList [("32",2),("7fe47286bc0adcd5cbd0a4013bc051d36773c9f0974b809ab4d6",2),("cda7455757cb034b1bd8321b567e2a6b71b13eeb",1)])])), mismatchExpected = MaryValue (Coin 35583235) (MultiAsset (fromList []))}))) :| []))))

Looking at the rendered deposit tx id we can see that indeed there are no tokens in the output:

"7c9d858e80fde98d61968ccabba46491ab965dbe97e4c8fb36e592ce7f15a97b"

== INPUTS (2)
- 73adac7c3908f8d14a872d525db39bf9772f56b64f99b5eb686708b1916bb07e#1
- e953a530ff8d02a9e7efbed8eadd0eccc7bb4e988f80da529e7186ef7a774e20#0
     ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "4ab68d8804c41aafe7c0bba0bc781a1497b53b430f39e552e71316af"})) StakeRefNull
     7512730 lovelace
     2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.32
     2 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.7fe47286bc0adcd5cbd0a4013bc051d36773c9f0974b809ab4d6
     1 dcf5fdd1d01c04b0e6262bba173a89c4b81b6570211f08bc059c8a75.cda7455757cb034b1bd8321b567e2a6b71b13eeb
     TxOutDatumNone
     ReferenceScriptNone

== COLLATERAL INPUTS (1)
- 73adac7c3908f8d14a872d525db39bf9772f56b64f99b5eb686708b1916bb07e#1

== REFERENCE INPUTS (0)

== OUTPUTS (3)
Total number of assets: 1
- ShelleyAddress Testnet (ScriptHashObj (ScriptHash "ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c")) StakeRefNull
     2000000 lovelace
     TxOutDatumInline [0,["0xe31c2cd8e640e8d8b902dcc3c4625ffea708e4a475189c06557b2225",1756914066710,[[0,[[0,["0xe953a530ff8d02a9e7efbed8eadd0eccc7bb4e988f80da529e7186ef7a774e20",0]],"0xd8799fd8799fd8799f581c4ab68d8804c41aafe7c0bba0bc781a1497b53b430f39e552e71316afffd87a80ffa140a1401a001e8480d87980d87a80ff"]]]]]
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "4ab68d8804c41aafe7c0bba0bc781a1497b53b430f39e552e71316af"})) StakeRefNull
     5512730 lovelace
     TxOutDatumNone
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "1052386136b347f3bb7c67fe3f2ee4ef120e1836e5d2707bb068afa6"})) StakeRefNull
     27881156 lovelace
     TxOutDatumNone

== TOTAL COLLATERAL
TxTotalCollateralNone

== RETURN COLLATERAL
TxReturnCollateralNone

== FEE
TxFeeExplicit ShelleyBasedEraConway (Coin 189349)

== VALIDITY
TxValidityNoLowerBound
TxValidityUpperBound ShelleyBasedEraConway (Just (SlotNo 37))

== MINT/BURN
0 lovelace

== SCRIPTS (0)
Total size (bytes):  0

== DATUMS (0)

== REDEEMERS (0)

== REQUIRED SIGNERS
[]

== METADATA
TxMetadataInEra ShelleyBasedEraConway (TxMetadata {unTxMetadata = fromList [(55555,TxMetaText "HydraV1/DepositTx")]})

Nice!
Now it is time to debug again and figure out where the problem is.
Seems like capUTxO is using only lovelace!
Yup that was the issue, not sure how it could get undetected like this.

August 2025

2025-08-14

SB on debugging H4 user issue on deposit

Finally it is time to put the visualize-logs executable to the test! I had to add apiTransactionTimeout to RunOptions in the logs since this user is using prior hydra-node version but after adding this I can see pretty logs ordered by timestamp.
The problem is the deposit and H4 that occurs on posting an increment tx.
I want to add ToPost lines to the logs so I can see and debug the deposit tx.
This is what was observed as deposited:

OnDepositTx {headId = UnsafeHeadId "\244\219p\156\221\"g\177\255\203\204\223\&6\212f\STXM\198t\228\&1\166\&8^\251:\164\171", depositTxId = "530ab09e96885ec211c3834059bde26ebf0ee3ec14826c6ece3eaf87decb5d91", deposited = fromList [(TxIn "164833d9c6456cc65305c441880ce5a1bfd47145f457839c2159c4ff85e15873" (TxIx 0),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "e06f2ae361f33815f775b224789025dccc4b6413599224e70841eebf"})) (StakeRefBase (KeyHashObj (KeyHash {unKeyHash = "eee790c7bb9c497f716fcec7d92a4d68c27b48c9e301b7c547c653cd"}))))) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 152000000) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "0836587ed7cee3c0790e24c930c67f31fc2511a3c25aa66ed205e05f"},fromList [("74525053",1000)]),(PolicyID {policyID = ScriptHash "9649407b4f02a38d98b1d9de2457eff522c47c87e22533377dcc70c4"},fromList [("446f6e67",10),("546f6b656e31",20),("55444f",10)])])))) TxOutDatumNone ReferenceScriptNone)], created = 2025-08-14 07:49:27 UTC, deadline = 2025-08-14 07:57:59.999 UTC}
OnDepositTx {headId = UnsafeHeadId "\244\219p\156\221\"g\177\255\203\204\223\&6\212f\STXM\198t\228\&1\166\&8^\251:\164\171", depositTxId = "530ab09e96885ec211c3834059bde26ebf0ee3ec14826c6ece3eaf87decb5d91", deposited = fromList [(TxIn "164833d9c6456cc65305c441880ce5a1bfd47145f457839c2159c4ff85e15873" (TxIx 0),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "e06f2ae361f33815f775b224789025dccc4b6413599224e70841eebf"})) (StakeRefBase (KeyHashObj (KeyHash {unKeyHash = "eee790c7bb9c497f716fcec7d92a4d68c27b48c9e301b7c547c653cd"}))))) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 152000000) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "0836587ed7cee3c0790e24c930c67f31fc2511a3c25aa66ed205e05f"},fromList [("74525053",1000)]),(PolicyID {policyID = ScriptHash "9649407b4f02a38d98b1d9de2457eff522c47c87e22533377dcc70c4"},fromList [("446f6e67",10),("546f6b656e31",20),("55444f",10)])])))) TxOutDatumNone ReferenceScriptNone)], created = 2025-08-14 07:49:27 UTC, deadline = 2025-08-14 07:57:59.999 UTC}

This is the deposit tx in the explorer https://preprod.cexplorer.io/tx/530ab09e96885ec211c3834059bde26ebf0ee3ec14826c6ece3eaf87decb5d91?tab=content
So what is locked should be 164833d9c6456cc65305c441880ce5a1bfd47145f457839c2159c4ff85e15873#0
Requested snapshot also looks correct:

utxoToCommit = Just (fromList [(TxIn "164833d9c6456cc65305c441880ce5a1bfd47145f457839c2159c4ff85e15873" (TxIx 0),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "e06f2ae361f33815f775b224789025dccc4b6413599224e70841eebf"})) (StakeRefBase (KeyHashObj (KeyHash {unKeyHash = "eee790c7bb9c497f716fcec7d92a4d68c27b48c9e301b7c547c653cd"}))))) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 152000000) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "0836587ed7cee3c0790e24c930c67f31fc2511a3c25aa66ed205e05f"},fromList [("74525053",1000)]),(PolicyID {policyID = ScriptHash "9649407b4f02a38d98b1d9de2457eff522c47c87e22533377dcc70c4"},fromList [("446f6e67",10),("546f6b656e31",20),("55444f",10)])])))) TxOutDatumNone ReferenceScriptNone)])

This is how increment tx looks like:

{
    "auxiliary scripts": null,
    "certificates": null,
    "collateral inputs": [],
    "currentTreasuryValue": null,
    "datums": [],
    "era": "Conway",
    "fee": "0 Lovelace",
    "governance actions": [],
    "inputs": [
        "02b2a8a41fa13712c675d069ac84cfc194cbbe2c4e4b6331289c9dbc747fbc98#0",
        "530ab09e96885ec211c3834059bde26ebf0ee3ec14826c6ece3eaf87decb5d91#0"
    ],
    "metadata": {
        "55555": "HydraV1/IncrementTx"
    },
    "mint": null,
    "outputs": [
        {
            "address": "addr_test1wzlxa0r5ggyvvc9lplwpe7a4z4680nfsthjmzam72awtknqz5cf7v",
            "address era": "Shelley",
            "amount": {
                "lovelace": 250094340,
                "policy 0836587ed7cee3c0790e24c930c67f31fc2511a3c25aa66ed205e05f": {
                    "asset 74525053 (tRPS)": 2000
                },
                "policy 9649407b4f02a38d98b1d9de2457eff522c47c87e22533377dcc70c4": {
                    "asset 446f6e67 (Dong)": 10,
                    "asset 546f6b656e31 (Token1)": 20,
                    "asset 55444f (UDO)": 10
                },
                "policy f4db709cdd2267b1ffcbccdf36d466024dc674e431a6385efb3aa4ab": {
                    "asset 3e31597e53ccb0a84c071a70f51ce38ec26175cb1d4671d0f6c5425c": 1,
                    "asset 4879647261486561645631 (HydraHeadV1)": 1,
                    "asset 58bc0a0b52009ff85b46233edd5966b2e0ae0524c4dc67f78af92396": 1,
                    "asset 67b499913a7169d504d7887c99b1ff862d980b6f3ad5683c6394cde0": 1
                }
            },
            "datum": {
                "constructor": 1,
                "fields": [
                    {
                        "constructor": 0,
                        "fields": [
                            {
                                "bytes": "f4db709cdd2267b1ffcbccdf36d466024dc674e431a6385efb3aa4ab"
                            },
                            {
                                "list": [
                                    {
                                        "bytes": "957698848d51dc52da009e088c85a6b85dd30230d279b7a468c2ac4f509ba227"
                                    },
                                    {
                                        "bytes": "7701af5180c40cb7cbdfdc2621dd4674166ff958e6d0e4d723003c3f6f0e553f"
                                    },
                                    {
                                        "bytes": "efd9a68d3dfe2981fe4672ecd8bfc764202310abfd3e4523209b162e14577b0e"
                                    }
                                ]
                            },
                            {
                                "constructor": 0,
                                "fields": [
                                    {
                                        "int": 60000
                                    }
                                ]
                            },
                            {
                                "int": 2
                            },
                            {
                                "bytes": "1e53176a1b9ae1620d8b1ddb536836ca3b8297e5b9ced616c408e8893b929b19"
                            }
                        ]
                    }
                ]
            },
            "network": "Testnet",
            "payment credential script hash": "be6ebc744208c660bf0fdc1cfbb5157477cd305de5b1777e575cbb4c",
            "reference script": null,
            "stake reference": null
        }
    ],
    "redeemers": [
        {
            "purpose": {
                "spending script witnessed input": "02b2a8a41fa13712c675d069ac84cfc194cbbe2c4e4b6331289c9dbc747fbc98#0"
            },
            "redeemer": {
                "data": "Constr 1 [Constr 0 [List [B \"\\RS\\ETX\\190e%\\153\\195X\\148\\186\\155\\175\\239\\192\\170z\\EOT\\n\\239\\EOT\\207L\\178-:+#v$*\\DC2\\204\\199%\\DC1J\\166\\224\\176\\213\\EM0\\NAK\\ACK\\191\\149\\206\\168\\148\\ENQ\\209z\\204\\209\\253<\\202\\254^\\157\\140\\204\\&4\\v\",B \"t\\141\\238\\n\\211=yH(\\234\\234F:\\173\\247\\178(\\EOT\\177\\229\\144`\\215\\EOT\\NAK\\US\\184o\\221\\222\\151\\214\\130\\197\\RS\\181\\215F\\185\\SI\\131\\191\\vA-1/\\153\\176\\181\\248\\174\\241\\169s\\RS\\134a\\255\\&2\\173#\\234\\r\",B \"r\\ESC0\\134T\\FS\\DC1y(\\USQ\\134#\\203\\DC3\\218`8`\\213I\\222l]\\132?\\ETB\\RS\\166z\\223\\182\\178A\\129\\EM[\\t\\221\\199f\\142k\\174\\137\\STX\\\\T\\226\\228\\&0\\205\\ESC\\162JR\\213\\231\\186\\188i#\\138\\ETX\"],I 624,Constr 0 [B \"S\\n\\176\\158\\150\\136^\\194\\DC1\\195\\131@Y\\189\\226n\\191\\SO\\227\\236\\DC4\\130ln\\206>\\175\\135\\222\\203]\\145\",I 0]]]",
                "execution units": {
                    "memory": 0,
                    "steps": 0
                }
            }
        },
        {
            "purpose": {
                "spending script witnessed input": "530ab09e96885ec211c3834059bde26ebf0ee3ec14826c6ece3eaf87decb5d91#0"
            },
            "redeemer": {
                "data": "Constr 0 [B \"\\244\\219p\\156\\221\\\"g\\177\\255\\203\\204\\223\\&6\\212f\\STXM\\198t\\228\\&1\\166\\&8^\\251:\\164\\171\"]",
                "execution units": {
                    "memory": 0,
                    "steps": 0
                }
            }
        }
    ],
    "reference inputs": [
        "48bd29e43dd01d12ab464f75fe40eed80e4051c8d3409e1cb20b8c01120b425e#0"
    ],
    "required signers (payment key hashes needed for scripts)": [
        "67b499913a7169d504d7887c99b1ff862d980b6f3ad5683c6394cde0"
    ],
    "return collateral": null,
    "scripts": [
        {
            "script data": {
                "plutus version": "PlutusV3",
                "script": "59044e59044b010100323232323232323232322533300332323232325332330093001300b37540042646644a66666602800c2646464a66601e6006002264a66602800201e264a666666032002020020020020264a66602c603200600a0226eb8004c058004c048dd50048a9998079803800899299980a000807899299999980c800808008008099299980b180c8018028089bad00101030160013012375401201c60206ea802054ccc034c004c03cdd5001099191919191929998099803980a9baa00d153330133300330044c103d87e80003371e6eb8c008c058dd50031bae30193016375401a2660066008980103d87980003322325333016300e30183754002266e24dd6980e180c9baa00100213300630074c103d87a80004a0600860306ea8c00cc060dd50011802980b1baa00e375a6002602c6ea8018528099299980a198021802a6103d87c80003322325333017300f30193754002266e20008dd6980e980d1baa00113300730084c103d87b80004a0600a60326ea8c014c064dd50011803180b9baa00f375a6004602e6ea801c4c8c8c8c8cc020c02530103d87d80003371e6e48ccc00ccc008ccc004004dd61802180d9baa0130052376600291010022337140040026e48ccc00ccc008c8cc004004dd61802980e1baa00c22533301e00114bd700991919800800998020021811801912999810800899811001a5eb804cc894ccc07ccdd79991192999811180d18121baa00113322533302433710004002298103d879800015333024337100020042980103d87b800014c103d87a8000375a6020604a6ea800cdd6980818129baa002100133225333023337200040022980103d8798000153330233371e0040022980103d87a800014c103d87b8000375c602060486ea8008dd7180818121baa001300e3022375400a601c60446ea800930103d8798000133024005003133024002330040040013023001302400130200012375c600e60386ea8005221002233714004002444a66603466e24005200014bd700a99980f0010a5eb804cc07cc080008ccc00c00cc084008cdc0000a400244646600200200644a66603c002297ae013301f37526006604000266004004604200244464666002002008006444a66603e004200226660060066044004660080026eb8c0840088c06cc070c0700045281bad30193016375401a4603260340024603000244a666024002294454cc04c008528119299980898028008a490344303100153330113009001149010344303200153330113370e90020008a490344303300153330113370e90030008a490344303400153330113370e90040008a49034430350014910344303600301237540024602a602c602c602c602c602c602c602c002602660206ea800854cc039241054c35353b350016370e900000580580580598080009808180880098061baa002370e90010b1806980700198060011805801180580098031baa00114984d95854cc0092401054c35313b3500165734ae7155ceaab9e5573eae815d0aba257481",
                "type": "plutus"
            },
            "script hash": "ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c"
        }
    ],
    "total collateral": null,
    "treasuryDonation": 0,
    "update proposal": null,
    "validity range": {
        "lower bound": null,
        "upper bound": 99474878
    },
    "voters": {},
    "withdrawals": null,
    "witnesses": []
}

I checked inputs/outputs and they look good so nothing really major that makes this increment tx fail. Let's dig deeper to see why would this tx fail.
I thought it might be related to how the datum is constructed but I see that there is an inline datum which is fine.
I can't tell why we get H4 . It is also not clear if the locked deposit hash matches which would reveal if we locked the commit wrong but that check happens after the failing one so we don't know.
I decoded the increment redeemer and found out that TxOutRef inside indeed matches the deposit input we are trying to spend.
User mentioned that if they used --contestation-period 60and --deposit-period 200 and if they use bigger values the problem goes away!
So how can it be that we see H4 and right after DepositExpired? H4 is unexpected here.

July 2025

2025-07-22

SB on fixing the input queue bug

After spending time to observe the test runs, debug and try to reason about the problem at hand I think that as the first step we should make sure that we can load persisted items on start. This can be achieved by just making sure that the queue capacity is large enough to handle all items. This is the most easy thing to do. We could also load incrementally and process items and then start but this requires more changes.
It feels like PersistentQueue is bolted on InputQueue which is not how we use PersistentQueue in Etcd module for example.
Perhaps it is worthwhile rewriting the createPersistInputQueue?
Let's tackle the first problem with making sure we can actually load all items from disk. I know this is not optimal but want to get results quick (tick-tack we don't have much time).
I also noticed we create the queue in race deep in some cps style code - why? Queue should be created only once?
Doing this makes the test red consistently
Next what we seemed to want to do in the test is to spam the node with many NewTx inputs and expect this to work out. This part was added:

            foldM_
              ( \utxo _i -> do
                  let Just (aliceTxIn, aliceTxOut) = UTxO.find (isVkTxOut aliceCardanoVk) utxo
                  let Right selfTx =
                        mkSimpleTx
                          (aliceTxIn, aliceTxOut)
                          (mkVkAddress testNetworkId aliceCardanoVk, txOutValue aliceTxOut)
                          aliceCardanoSk
                  send node $ input "NewTx" ["transaction" .= selfTx]
                  pure utxo
              )
              initialUTxO
              [1 .. capacity * 10]

and then we get:

uncaught exception: IOException of type ResourceVanished          
writev: resource vanished (Connection reset by peer)

So it seems this kills hydra-node

If I just spam the node without exceeding the queue capacity I get:

 test/Test/EndToEndSpec.hs:820:11:
  1) Test.EndToEnd, End-to-end on Cardano devnet, withHydraNode, load persistent queue with capacity exceeded
       hydra-node (nodeId = 1) exited with failure code: 1
       thread blocked indefinitely in an STM transaction
       CallStack (from HasCallStack):
         wrapBlockedIndefinitely, called at src/Control/Monad/Class/MonadSTM/Internal.hs:546:16 in io-classes-1.5.0.0-CQ6SF7CN7ju8okIkiv0F3i:Control.Monad.Class.MonadSTM.Internal
         atomically, called at src/Hydra/PersistentQueue.hs:91:3 in hydra-node-0.22.2-inplace:Hydra.PersistentQueue
         writePersistentQueue, called at src/Hydra/Node/InputQueue.hs:97:19 in hydra-node-0.22.2-inplace:Hydra.Node.InputQueue

We use offline chain config in the test and we are not opening the head but still send a bunch of NewTx inputs. I'll refactor the test so that we have the head opened so to make sure we don't get any exceptions that could kill the queue thread.
This change makes the test green even with 10x capacity exceeded.

2025-07-03

SB on increment bug

Ok so it seems like when two parties commit nothing to a Head and then one tries to increment with 1 ada (contestation-period 300s, deposit-deadline 600s) the increment transaction fails with H4 HeadValueIsNotPreserved.
I'll walk trough the code to try and reason out why this happens.
Let's go backwards from the increment tx construction.
The increment transaction looks like this:

{
    "auxiliary scripts": null,
    "certificates": null,
    "collateral inputs": [],
    "currentTreasuryValue": null,
    "era": "Conway",
    "fee": "0 Lovelace",
    "governance actions": [],
    "inputs": [
        "ac6ab83cf1a766fedc68e26ca33dbd5f457ecbb00997f34bf178f757d7152c81#0",
        "cf95ebccdc7cc42eb8957e3f34c96722f9022ea9e0a2dc64e30e82d342990de4#0"
    ],
    "metadata": {
        "55555": "HydraV1/IncrementTx"
    },
    "mint": null,
    "outputs": [
        {
            "address": "addr_test1wzlxa0r5ggyvvc9lplwpe7a4z4680nfsthjmzam72awtknqz5cf7v",
            "address era": "Shelley",
            "amount": {
                "lovelace": 5663420,
                "policy 6714b1424806d7436033096cfe829fef534436c6f98992a4bd0f43ae": {
                    "asset 4879647261486561645631 (HydraHeadV1)": 1,
                    "asset 4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b41": 1,
                    "asset 5ecf89bf0e00eaf8667d9ff9bb92c6faac2fbcc6ff71ce9a8d6c5033": 1
                }
            },
            "datum": {
                "constructor": 1,
                "fields": [
                    {
                        "constructor": 0,
                        "fields": [
                            {
                                "bytes": "6714b1424806d7436033096cfe829fef534436c6f98992a4bd0f43ae"
                            },
                            {
                                "list": [
                                    {
                                        "bytes": "a65dd1af5d938e394b05198b97870a33343d3dd2ec8feb8906a2cbbd2cc0ac96"
                                    },
                                    {
                                        "bytes": "32bce7668b68bcc66750963217e50244fb7a9495053fb787fd5480aa1e509366"
                                    }
                                ]
                            },
                            {
                                "constructor": 0,
                                "fields": [
                                    {
                                        "int": 300000
                                    }
                                ]
                            },
                            {
                                "int": 1
                            },
                            {
                                "bytes": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
                            }
                        ]
                    }
                ]
            },
            "network": "Testnet",
            "payment credential script hash": "be6ebc744208c660bf0fdc1cfbb5157477cd305de5b1777e575cbb4c",
            "reference script": null,
            "stake reference": null
        }
    ],
    "redeemers": [
        {
            "purpose": {
                "spending script witnessed input": "ac6ab83cf1a766fedc68e26ca33dbd5f457ecbb00997f34bf178f757d7152c81#0"
            },
            "redeemer": {
                "data": "Constr 0 [B \"g\\DC4\\177BH\\ACK\\215C`3\\tl\\254\\130\\159\\239SD6\\198\\249\\137\\146\\164\\189\\SIC\\174\"]",
                "execution units": {
                    "memory": 0,
                    "steps": 0
                }
            }
        },
        {
            "purpose": {
                "spending script witnessed input": "cf95ebccdc7cc42eb8957e3f34c96722f9022ea9e0a2dc64e30e82d342990de4#0"
            },
            "redeemer": {
                "data": "Constr 1 [Constr 0 [List [B \"\\143\\&5\\138P\\189\\233\\\\\\230\\156\\NAK\\bI\\176\\137\\178\\233r\\185\\202\\156\\134%?:\\219p\\178\\213\\&8\\204\\r\\179I<\\NUL\\236\\228\\255~\\199\\229\\194\\232\\207\\243\\200\\ETB\\138\\143\\f1H:k~\\f$\\209\\219a\\221\\246\\NUL\\EOT\",B \"\\163\\163\\CAN<='\\150\\FS\\254PaS\\EMNA\\180\\131\\185\\219V3\\176\\156\\173\\208\\189U\\160A\\216\\209\\v\\225\\f'\\241\\153\\ACK\\135g\\247usn\\STX\\SO\\US\\209\\172UO\\191\\207Y\\164\\176t2i\\228\\165\\212w\\b\"],I 1,Constr 0 [B \"\\172j\\184<\\241\\167f\\254\\220h\\226l\\163=\\189_E~\\203\\176\\t\\151\\243K\\241x\\247W\\215\\NAK,\\129\",I 0]]]",
                "execution units": {
                    "memory": 0,
                    "steps": 0
                }
            }
        }
    ],
    "reference inputs": [
        "39924b477411b6bd33258f63dd705e0712b513a21ea9bc422a23ec0be797ba03#0"
    ],
    "required signers (payment key hashes needed for scripts)": [
        "4fb53f7aa3e07e3f3ba8516ed345119e9841822385c0307311209b41"
    ],
    "return collateral": null,
    "total collateral": null,
    "treasuryDonation": 0,
    "update proposal": null,
    "validity range": {
        "lower bound": null,
        "upper bound": 84879619
    },
    "voters": {},
    "withdrawals": null,
    "witnesses": []
}

And these are the transaction inputs: https://preview.cexplorer.io/tx/ac6ab83cf1a766fedc68e26ca33dbd5f457ecbb00997f34bf178f757d7152c81 https://preview.cexplorer.io/tx/cf95ebccdc7cc42eb8957e3f34c96722f9022ea9e0a2dc64e30e82d342990de4

So these are collect com and deposit outputs spent in the increment tx and deposit output needs to end up in the single Head output.
I noticed that we tried to lock 1 ADA but the output contains 1.53 ada so the theory is that the 1 ada get's autobalanced and locked into a deposit contract as 1.53 ada while offchain code thinks the locked value is just 1 ada. This causes H4 later on since the amounts do not match.
We tested this assumption and it proves correct! All we need to do now is reject too low deposits so we don't scratch our heads again about this.

June 2025

2025-06-19

SB on debugging user logs

Loaded two node logs in vim and I filtered out etcd messages.
This is the deposit they tried to do https://preview.cardanoscan.io/transaction/4dea5f74880a761a3d312e122b9701e9ad2ed2d74880de5b47bef087ee10de14?tab=utxo
The user experienced H4 - HeadValueIsNotPreserved upon trying to post increment that looks like this:

{
    "auxiliary scripts": null,
    "certificates": null,
    "collateral inputs": [],
    "currentTreasuryValue": null,
    "era": "Conway",
    "fee": "0 Lovelace",
    "governance actions": [],
    "inputs": [
        "4dea5f74880a761a3d312e122b9701e9ad2ed2d74880de5b47bef087ee10de14#0",
        "843676b219d25b597ae6f74dfc3a14ff11996f5e4bc282db2e847837e6abff4d#0"
    ],
    "metadata": {
        "55555": "HydraV1/IncrementTx"
    },
    "mint": null,
    "outputs": [
        {
            "address": "addr_test1wzlxa0r5ggyvvc9lplwpe7a4z4680nfsthjmzam72awtknqz5cf7v",
            "address era": "Shelley",
            "amount": {
                "lovelace": 6107270,
                "policy 361b4ebba480bf5f279e513401bd1eb1bb1b1117000f9876d0b84401": {
                    "asset 546573744e4654 (TestNFT)": 1
                },
                "policy 5ea251107632c490977d94ba46f2d5466aac9ce643bdc87e2b61e8e3": {
                    "asset 14d26549b376a0a7b8ed8449948b28bd00d179527edfe57c6b5c5859": 1,
                    "asset 4879647261486561645631 (HydraHeadV1)": 1,
                    "asset ff6fd6496610513e984c82c611866d052e8183bba8e09db1c2fb6688": 1
                }
            },
            "datum": {
                "constructor": 1,
                "fields": [
                    {
                        "constructor": 0,
                        "fields": [
                            {
                                "bytes": "5ea251107632c490977d94ba46f2d5466aac9ce643bdc87e2b61e8e3"
                            },
                            {
                                "list": [
                                    {
                                        "bytes": "ca64297d89d65052b090bde90e5a5f97b56e572187157c7360900257d02a0f38"
                                    },
                                    {
                                        "bytes": "8c6e2f026318fc925cfd6de9913ba71325a805e95d2f8d258e2e2f82baf66c0b"
                                    }
                                ]
                            },
                            {
                                "constructor": 0,
                                "fields": [
                                    {
                                        "int": 300000
                                    }
                                ]
                            },
                            {
                                "int": 1
                            },
                            {
                                "bytes": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
                            }
                        ]
                    }
                ]
            },
            "network": "Testnet",
            "payment credential script hash": "be6ebc744208c660bf0fdc1cfbb5157477cd305de5b1777e575cbb4c",
            "reference script": null,
            "stake reference": null
        }
    ],
    "redeemers": [
        {
            "purpose": {
                "spending script witnessed input": "4dea5f74880a761a3d312e122b9701e9ad2ed2d74880de5b47bef087ee10de14#0"
            },
            "redeemer": {
                "data": "Constr 0 [B \"^\\162Q\\DLEv2\\196\\144\\151}\\148\\186F\\242\\213Fj\\172\\156\\230C\\189\\200~+a\\232\\227\"]",
                "execution units": {
                    "memory": 0,
                    "steps": 0
                }
            }
        },
        {
            "purpose": {
                "spending script witnessed input": "843676b219d25b597ae6f74dfc3a14ff11996f5e4bc282db2e847837e6abff4d#0"
            },
            "redeemer": {
                "data": "Constr 1 [Constr 0 [List [B \"\\180\\t\\SUB\\b\\210\\179\\203\\238\\180\\FS[\\f\\175\\b\\199\\211\\CAN\\221/ptu__\\161\\188\\n1|\\131Pd\\252`\\130<wN\\183\\187\\182\\210)\\150\\214\\147(C\\232\\173\\157\\153r\\235\\174\\188\\154\\177\\208\\241\\254\\DC4\\227\\ENQ\",B \"\\183\\250\\219\\217\\133\\138\\128\\250,\\199\\&0v\\175\\215\\161\\238\\SUB2u\\175\\231\\165\\SI\\EM\\218\\137\\246\\NUL\\254\\172W\\STX\\250\\162\\250\\234\\177\\209\\CAN\\129m\\f>xW2\\177\\225\\204'\\235/\\150\\184\\225\\SOF\\SI\\216\\181{\\188W\\a\"],I 1,Constr 0 [B \"M\\234_t\\136\\nv\\SUB=1.\\DC2+\\151\\SOH\\233\\173.\\210\\215H\\128\\222[G\\190\\240\\135\\238\\DLE\\222\\DC4\",I 0]]]",
                "execution units": {
                    "memory": 0,
                    "steps": 0
                }
            }
        }
    ],
    "reference inputs": [
        "39924b477411b6bd33258f63dd705e0712b513a21ea9bc422a23ec0be797ba03#0"
    ],
    "required signers (payment key hashes needed for scripts)": [
        "14d26549b376a0a7b8ed8449948b28bd00d179527edfe57c6b5c5859"
    ],
    "return collateral": null,
    "total collateral": null,
    "treasuryDonation": 0,
    "update proposal": null,
    "validity range": {
        "lower bound": null,
        "upper bound": 83567522
    },
    "voters": {},
    "withdrawals": null,
    "witnesses": []
}

The input utxo look like this:

            "843676b219d25b597ae6f74dfc3a14ff11996f5e4bc282db2e847837e6abff4d#0": {
              "address": "addr_test1wzlxa0r5ggyvvc9lplwpe7a4z4680nfsthjmzam72awtknqz5cf7v",
              "datum": null,
              "referenceScript": null,
              "value": {
                "5ea251107632c490977d94ba46f2d5466aac9ce643bdc87e2b61e8e3": {
                  "14d26549b376a0a7b8ed8449948b28bd00d179527edfe57c6b5c5859": 1,
                  "4879647261486561645631": 1,
                  "ff6fd6496610513e984c82c611866d052e8183bba8e09db1c2fb6688": 1
                },
                "lovelace": 4663420
              }
            }
            "4dea5f74880a761a3d312e122b9701e9ad2ed2d74880de5b47bef087ee10de14#0": {
                "address": "addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j",
                "datum": null,
                "referenceScript": null,
                "value": {
                  "361b4ebba480bf5f279e513401bd1eb1bb1b1117000f9876d0b84401": {
                    "546573744e4654": 1
                  },
                  "lovelace": 2348950
                }
              },

So this is the deposit tx https://preview.cardanoscan.io/transaction/4dea5f74880a761a3d312e122b9701e9ad2ed2d74880de5b47bef087ee10de14?tab=utxo
We see that the locked up UTxO is the one coming from the snapshot too so how come there is this one 4dea5f74880a761a3d312e122b9701e9ad2ed2d74880de5b47bef087ee10de14 used as the input? I think that since we construct the increment from the spendableUTxOfor some reason we find the wrong one which also means the observation could contain some bug but I want to be able to figure all this out from the logs.

May 2025

2025-05-19

SB on looking at the user issue with committing

We have a user having problems with committing a script into a head incrementally. They commit empty UTxO for two parties and then try to increment using a blueprint tx which leads to them unable to close and fanout.
They see H4 on increment tx HeadValueIsNotPreserved and then on fanout this leads to H54 FanoutUTxOToCommitHashMismatch but that should be the outcome of wrong increment.
First I want to take a look at the logs and detect the state of things before the actual increment takes place.
This is the deposit of node A together with the spendable utxo:

Deposit A

{
  "timestamp": "2025-05-15T08:27:38.493319047Z",
  "threadId": 92,
  "namespace": "HydraNode-\"1\"",
  "message": {
    "node": {
      "by": {
        "vkey": "0227964e8ce2091ab78f431bfa8c468ae7edb58b6626830847e75f2b4fb0f064"
      },
      "input": {
        "chainEvent": {
          "newChainState": {
            "recordedAt": {
              "blockHash": "dc9a6d8cd0b2b7f199ee524df54777e021fb69cef73f66b453c6e5ff27ec348e",
              "slot": 80641658,
              "tag": "ChainPoint"
            },
            "spendableUTxO": {
              "58a19e03bfc97f79b6d99f9a1ddc21d19f67d6f0ee2b717cea3bfea506207a2f#0": {
                "address": "addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j",
                "datum": null,
                "inlineDatum": {
                  "constructor": 0,
                  "fields": [
                    {
                      "bytes": "c3d43601333ee3b9904de784b11cb51848f121849dd918d33ce17e0a"
                    },
                    {
                      "int": 1747297945581
                    },
                    {
                      "list": [
                        {
                          "constructor": 0,
                          "fields": [
                            {
                              "constructor": 0,
                              "fields": [
                                {
                                  "bytes": "92328d0e0f67fb11a62155dc6ca3b9c8905e7afadb87d67231cbce9613d9cc6a"
                                },
                                {
                                  "int": 0
                                }
                              ]
                            },
                            {
                              "bytes": "d8799fd8799fd8799f581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261effd8799fd8799fd8799f581cea1a07cebb935acfe6fe49e9cfbd2864e21e3d6385a4391d29782f5cffffffffa240a1401a0016080a581c41a1a2139e297b0af14d55506569157a6a258c49ce2907bf7ec78b54a147546573744e465401d87b9fd8799fa2435f706b581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261e446e616d654d54657374204d696e74204e465401ffffd87a80ff"
                            }
                          ]
                        }
                      ]
                    }
                  ]
                },
                "inlineDatumRaw": "d8799f581cc3d43601333ee3b9904de784b11cb51848f121849dd918d33ce17e0a1b00000196d312c7ed9fd8799fd8799f582092328d0e0f67fb11a62155dc6ca3b9c8905e7afadb87d67231cbce9613d9cc6a00ff5f5840d8799fd8799fd8799f581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261effd8799fd8799fd8799f581cea1a07cebb935acfe6fe49e9cf5840bd2864e21e3d6385a4391d29782f5cffffffffa240a1401a0016080a581c41a1a2139e297b0af14d55506569157a6a258c49ce2907bf7ec78b54a1475465737458404e465401d87b9fd8799fa2435f706b581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261e446e616d654d54657374204d696e74204e46544701ffffd87a80ffffffffff",
                "inlineDatumhash": "7072708fc774009787ea7ce1d0d4398fd99f0485f443cacd293086259e27f922",
                "referenceScript": null,
                "value": {
                  "41a1a2139e297b0af14d55506569157a6a258c49ce2907bf7ec78b54": {
                    "546573744e4654": 1
                  },
                  "lovelace": 2348950
                }
              },
              "58a19e03bfc97f79b6d99f9a1ddc21d19f67d6f0ee2b717cea3bfea506207a2f#1": {
                "address": "addr_test1vqg3qe4umvdju0nxjs77rcv98rzmk4z9cshced8ysz520dqta4a9t",
                "datum": null,
                "datumhash": null,
                "inlineDatum": null,
                "inlineDatumRaw": null,
                "referenceScript": null,
                "value": {
                  "lovelace": 990666095
                }
              },
              "79d547486be78ea86594dc9e4bc855df4ae42472ed375f4555b309ff0ee72055#0": {
                "address": "addr_test1wq8r2y26937p835wekxhfeyc0szdg5u7xdmy803qhve8f0gsqwq93",
                "datum": null,
                "inlineDatum": {
                  "constructor": 1,
                  "fields": [
                    {
                      "constructor": 0,
                      "fields": [
                        {
                          "bytes": "c3d43601333ee3b9904de784b11cb51848f121849dd918d33ce17e0a"
                        },
                        {
                          "list": [
                            {
                              "bytes": "0227964e8ce2091ab78f431bfa8c468ae7edb58b6626830847e75f2b4fb0f064"
                            },
                            {
                              "bytes": "c5c1e857c0dbad1fb55a51acb46ad753889bd4aabb4cf1d5831a5379fe3af48b"
                            }
                          ]
                        },
                        {
                          "constructor": 0,
                          "fields": [
                            {
                              "int": 300000
                            }
                          ]
                        },
                        {
                          "int": 0
                        },
                        {
                          "bytes": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
                        }
                      ]
                    }
                  ]
                },
                "inlineDatumRaw": "d87a9fd8799f581cc3d43601333ee3b9904de784b11cb51848f121849dd918d33ce17e0a9f58200227964e8ce2091ab78f431bfa8c468ae7edb58b6626830847e75f2b4fb0f0645820c5c1e857c0dbad1fb55a51acb46ad753889bd4aabb4cf1d5831a5379fe3af48bffd8799f1a000493e0ff005820e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855ffff",
                "inlineDatumhash": "17460f2bd89e876a0d6bfb75f10971262c3d7c9dd5bb3d382f75edf814127c0a",
                "referenceScript": null,
                "value": {
                  "c3d43601333ee3b9904de784b11cb51848f121849dd918d33ce17e0a": {
                    "0861f42c304f2eaecf578f4631684dc338749ddd1a556d8bf8eed360": 1,
                    "111066bcdb1b2e3e66943de1e18538c5bb5445c42f8cb4e480a8a7b4": 1,
                    "4879647261486561645631": 1
                  },
                  "lovelace": 4663420
                }
              },
              "79d547486be78ea86594dc9e4bc855df4ae42472ed375f4555b309ff0ee72055#1": {
                "address": "addr_test1vqyxrapvxp8jatk02785vvtgfhpnsayam5d92mvtlrhdxcqp4jnw7",
                "datum": null,
                "datumhash": null,
                "inlineDatum": null,
                "inlineDatumRaw": null,
                "referenceScript": null,
                "value": {
                  "lovelace": 996105253
                }
              }
            }
          },
          "observedTx": {
            "deadline": "2025-05-15T08:32:25.581Z",
            "depositTxId": "58a19e03bfc97f79b6d99f9a1ddc21d19f67d6f0ee2b717cea3bfea506207a2f",
            "deposited": {
              "92328d0e0f67fb11a62155dc6ca3b9c8905e7afadb87d67231cbce9613d9cc6a#0": {
                "address": "addr_test1qzzgryrnvj9qtv4p0wnx5dle4k7a3z99jzxsh0p20ajzv8h2rgruawuntt87dljfa88m62ryug0r6cu95su362tc9awq660nfq",
                "datum": null,
                "inlineDatum": {
                  "constructor": 0,
                  "fields": [
                    {
                      "map": [
                        {
                          "k": {
                            "bytes": "5f706b"
                          },
                          "v": {
                            "bytes": "84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261e"
                          }
                        },
                        {
                          "k": {
                            "bytes": "6e616d65"
                          },
                          "v": {
                            "bytes": "54657374204d696e74204e4654"
                          }
                        }
                      ]
                    },
                    {
                      "int": 1
                    }
                  ]
                },
                "inlineDatumRaw": "d8799fa2435f706b581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261e446e616d654d54657374204d696e74204e465401ff",
                "inlineDatumhash": "2e733e63e1f1f8231eee004ce50670a90624b6b0a0d16260f92a485a1a5d86a1",
                "referenceScript": null,
                "value": {
                  "41a1a2139e297b0af14d55506569157a6a258c49ce2907bf7ec78b54": {
                    "546573744e4654": 1
                  },
                  "lovelace": 1443850
                }
              }
            },
            "headId": "c3d43601333ee3b9904de784b11cb51848f121849dd918d33ce17e0a",
            "tag": "OnDepositTx"
          },
          "tag": "Observation"
        },
        "tag": "ChainInput"
      },
      "inputId": 39,
      "tag": "BeginInput"
    },
    "tag": "Node"
  }
}

and the same on preview explorer https://preview.cardanoscan.io/transaction/58a19e03bfc97f79b6d99f9a1ddc21d19f67d6f0ee2b717cea3bfea506207a2f?tab=utxo

It seems there is a difference in what we observe as deposit and the spendable utxo we have locally. We observe a deposit of:

{
              "92328d0e0f67fb11a62155dc6ca3b9c8905e7afadb87d67231cbce9613d9cc6a#0": {
                "address": "addr_test1qzzgryrnvj9qtv4p0wnx5dle4k7a3z99jzxsh0p20ajzv8h2rgruawuntt87dljfa88m62ryug0r6cu95su362tc9awq660nfq",
                "datum": null,
                "inlineDatum": {
                  "constructor": 0,
                  "fields": [
                    {
                      "map": [
                        {
                          "k": {
                            "bytes": "5f706b"
                          },
                          "v": {
                            "bytes": "84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261e"
                          }
                        },
                        {
                          "k": {
                            "bytes": "6e616d65"
                          },
                          "v": {
                            "bytes": "54657374204d696e74204e4654"
                          }
                        }
                      ]
                    },
                    {
                      "int": 1
                    }
                  ]
                },
                "inlineDatumRaw": "d8799fa2435f706b581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261e446e616d654d54657374204d696e74204e465401ff",
                "inlineDatumhash": "2e733e63e1f1f8231eee004ce50670a90624b6b0a0d16260f92a485a1a5d86a1",
                "referenceScript": null,
                "value": {
                  "41a1a2139e297b0af14d55506569157a6a258c49ce2907bf7ec78b54": {
                    "546573744e4654": 1
                  },
                  "lovelace": 1443850
                }
              }
            },

https://preview.cexplorer.io/tx/92328d0e0f67fb11a62155dc6ca3b9c8905e7afadb87d67231cbce9613d9cc6a

But the locked deposit looks like this one:

              "58a19e03bfc97f79b6d99f9a1ddc21d19f67d6f0ee2b717cea3bfea506207a2f#0": {
                "address": "addr_test1wzhqrkk782wrgm2ujwhreceegy4epg9clqlefmrt4gjwxrqckxj2j",
                "datum": null,
                "inlineDatum": {
                  "constructor": 0,
                  "fields": [
                    {
                      "bytes": "c3d43601333ee3b9904de784b11cb51848f121849dd918d33ce17e0a"
                    },
                    {
                      "int": 1747297945581
                    },
                    {
                      "list": [
                        {
                          "constructor": 0,
                          "fields": [
                            {
                              "constructor": 0,
                              "fields": [
                                {
                                  "bytes": "92328d0e0f67fb11a62155dc6ca3b9c8905e7afadb87d67231cbce9613d9cc6a"
                                },
                                {
                                  "int": 0
                                }
                              ]
                            },
                            {
                              "bytes": "d8799fd8799fd8799f581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261effd8799fd8799fd8799f581cea1a07cebb935acfe6fe49e9cfbd2864e21e3d6385a4391d29782f5cffffffffa240a1401a0016080a581c41a1a2139e297b0af14d55506569157a6a258c49ce2907bf7ec78b54a147546573744e465401d87b9fd8799fa2435f706b581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261e446e616d654d54657374204d696e74204e465401ffffd87a80ff"
                            }
                          ]
                        }
                      ]
                    }
                  ]
                },
                "inlineDatumRaw": "d8799f581cc3d43601333ee3b9904de784b11cb51848f121849dd918d33ce17e0a1b00000196d312c7ed9fd8799fd8799f582092328d0e0f67fb11a62155dc6ca3b9c8905e7afadb87d67231cbce9613d9cc6a00ff5f5840d8799fd8799fd8799f581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261effd8799fd8799fd8799f581cea1a07cebb935acfe6fe49e9cf5840bd2864e21e3d6385a4391d29782f5cffffffffa240a1401a0016080a581c41a1a2139e297b0af14d55506569157a6a258c49ce2907bf7ec78b54a1475465737458404e465401d87b9fd8799fa2435f706b581c84819073648a05b2a17ba66a37f9adbdd888a5908d0bbc2a7f64261e446e616d654d54657374204d696e74204e46544701ffffd87a80ffffffffff",
                "inlineDatumhash": "7072708fc774009787ea7ce1d0d4398fd99f0485f443cacd293086259e27f922",
                "referenceScript": null,
                "value": {
                  "41a1a2139e297b0af14d55506569157a6a258c49ce2907bf7ec78b54": {
                    "546573744e4654": 1
                  },
                  "lovelace": 2348950
                }
              },

This means when increment is constructed using spendable utxo it is just wrong and we get a mismatch between values we expect in the head input + deposit and the head output. But the question is why do we try to construct the increment with wrong UTxO?

Deposit observation does not depend on the spendable UTxO at all, it cares only about the deposit transaction which then leads to posting increment tx which looks at spendable UTxO and tries to find some deposit utxo to spend. Clearly this leads to problems if we have different UTxO in the spendable utxo and observation!
Thinking on how to reproduce this in some test. I wonder how did we end up with some deposit utxo in our spendable/Head UTxO? Perhaps we need to do what SN suggested and tie together tx with UTxO - what we call ResolvedTx? That way the observed UTxO is always tied to some tx we observed.

2025-05-13

SB on hydra-node using blockfrost

Time to finalize the blockfrost PR. I still need to see fanout work but I expect no problems.
There are couple of ideas related to improving the design of this integration. I'll look up adr3 or so and if that sounds like it will take too much time I'll default to having two instances of BackendOps - one for direct and one for blockfrost backend.
I don't see close tx being on chain. Need to inspect.
Yup the close tx is never ending up on chain for some reason and I don't see any errors. Wat? I did see a deposit observation in between which is not related to this head. Simplifying the snapshot to only close whatever we committed I think is fine so I'll do that. Waiting on the test results is really taking it's time.
So the tx did not even end up in mempool! Wat? How come I don't see any error?
Perhaps the submission client stopped and is not submitting txs silently? But I saw PostedTx for this close already.
Waiting for a long time yielded this error:

        uncaught exception: APIBlockfrostError
       BlockfrostError "ServantClientError (ConnectionError (HttpExceptionRequest Request {\n  host                 = \"cardano-preview.blockfrost.io\"\n  port                 = 443\n  secure               = True\n  requestHeaders       = [(\"Accept\",\"application/json;charset=utf-8,application/json\"),(\"User-Agent\",\"blockfrost-haskell/0.12.2.0\"),(\"project_id\",\"w25SicgUreG35eWDNnSx6ZgYhvPrRpS8\")]\n  path                 = \"/api/v0/blocks/26ae9bb9162c694c7aac69dcfbcff931f260a17badc1e3cb811f7698af8ef1da\"\n  queryString          = \"\"\n  method               = \"GET\"\n  proxy                = Nothing\n  rawBody              = False\n  redirectCount        = 10\n  responseTimeout      = ResponseTimeoutDefault\n  requestVersion       = HTTP/1.1\n  proxySecureMode      = ProxySecureWithConnect\n}\n (InternalException (HandshakeFailed (Error_Misc \"Network.Socket.recvBuf: resource vanished (Connection reset by peer)\")))))"

so it seems blockfrost chokes sometimes...
I'll just wait for the test to either finish or die since I don't have any bright ideas on what could cause this problem with close when all other txs go through and are observed.
Maybe I do the typeclass changes and have somebody from the team to look at this with me? I am kinda tired of doing this thing for so long.
I gave up on the idea since I am not satisfied with these two separate instances. I used either type for the chain backend in the Options and this is not making it look pretty. Am I missing something? Would like to work with someone on this since I don't want to loose any more time.
Looking at close again I figured out what can be the problem - tx validity. Indeed the close tx starting validity looks like this:

TxValidityLowerBound AllegraEraOnwardsConway (SlotNo 80486130)
TxValidityUpperBound ShelleyBasedEraConway (Just (SlotNo 80486140))

but the tick immediately after submitting close tx is:

Tick
    { chainTime = 2025-05-13 13:15:25 UTC
    , chainSlot = ChainSlot 80486125
    }

So I can't get the exact slot of posted tx but what I believe is that the close tx get's discarded because of it's validity range and the tx never actually ends up on-chain. This would explain why I don't see it in the explorer as well.

Hypothesis was right! Bumping the contestation deadline as this is responsible for calculating the end validity of the transaction did the trick. I finally have a green test!

2025-05-12

SB on hydra-node using blockfrost

Today I want to get to a working test finally and for that to work out I need to find a bug in the observation code that prevents me from observing commit transaction.
I've sprinkled some tracing in the observation code to be able to figure out why a tx was not observed.
I see that the observation check that we are spending the initial output is failing for whatever reason.
This happens because resolving input UTxO comes out empty. But why exactly?
Perhaps I need to wait to see the published scripts first before observing but that doesn't make too much sense to me. TxIds I pass into direct chain should then not be valid and I should get a different error? Anyway, I'll give it a shot.
Nah that was not it. I noticed one important thing and that is that the slots keep going back. So, observed slots should always go forward otherwise we will update the spendable utxo again and rewind it back to the init observation state and not be able to observe next ones.
This is definitely a bug I am experiencing and I can't tell for now why do we go back to the first starting slot after observing the init?
Do I need to implement onRollBackward too? I can't see where do I go back to already seen slot argh
Just by using a TVar to record a block hash things started working! Wat? I was under the impression I am using the recursion correctly? Is it the retry that was screwing things up?
Now got a fuel utxo error related to collect yay!
Duh, I was using Alice keys for everything, fixed the error by using external generated key for committing and actually having enough funds.

2025-05-08

SB on hydra-node using blockfrost

I need to see why I can't query by address and txId and get UTxO for both faucet and alice?
Ok, this was happening because there is a LOT of UTxO at both faucet and alice address. Blockfrost gives us max 100 entries sorted asc so I couldn't find the last UTxO.
I've changed the awaitUTxO query to return only the last UTxO for specific address and then filter by txid.
This worked and I got further! Now the commit tx fails because of insufficient collateral.
This seems random and it worked second try? I added more ada and now it seems to work but then observing commit failed with:

        expected: OnCommitTx {headId = UnsafeHeadId "\212\164\159K\128/\130j\169\170\211\155\186\128}\182z\205KC\STX\162;b;K\"\DC3", party = Party {vkey = HydraVerificationKey (VerKeyEd25519DSIGN "d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4")}, committed = fromList [(TxIn "9bae27ff84d84af2a808432341801edab64c75dc8ce38664b09ee54fa8abfa2f" (TxIx 0),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d"})) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 7000000) (MultiAsset (fromList [])))) TxOutDatumNone ReferenceScriptNone)]}
        but got: OnInitTx {headId = UnsafeHeadId "\212\164\159K\128/\130j\169\170\211\155\186\128}\182z\205KC\STX\162;b;K\"\DC3", headSeed = UnsafeHeadSeed "\"b511295ef9050a5f4df3c52029abce4c9e3dd703bc6e606746640f46aca5679c#0\"", headParameters = HeadParameters {contestationPeriod = 10s, parties = [Party {vkey = HydraVerificationKey (VerKeyEd25519DSIGN "d5bf4a3fcce717b0388bcc2749ebc148ad9969b23f45ee1b605fd58778576ac4")}]}, participants = [UnsafeOnChainId "\248\166\140\209\142Y\166\172\232H\NAKZ\SO\150z\246OM\NUL\207\138\206\232\173\201Zk\r"]}

I already wait for 100 seconds which is way too much but perhaps I should wait some more to see if the observation goes through.
Bumping to 150 yields the same error so perhaps I need to look at observation code. Feels weird that we would observe init but fail to observe commit. Problems never stop appearing it seems.
Since we are just submitting a commit tx and then expect to see the observation I added the code to actually await for the utxo to appear at the address before waiting on the observation.
I also added a fail after some number of seconds when awaiting for utxo since we don't want to have tests running forever.
Now I see unexpected deposit observation after commit which means I need to wait some more for the deposit to settle.
Waiting more didn't help - I am confused since I can observe InitTx just fine and this observation appears again when I am trying to observe Commit after sending external commit tx.
On a separate note - if I run the test, ctrl-c it and run again the faucet utxo is not correct since we didn't reobserve it. There should be a way of knowing if certain UTxO is likely the last one and there is no pending tx in progress for certain address. I need to explore blockfrost api to see if I can find something suitable (mempool diagnostics or similar)
Nice there is https://docs.blockfrost.io/#tag/cardano--mempool/GET/mempool/addresses/{address} and https://docs.blockfrost.io/#tag/cardano--mempool/GET/mempool/{hash}
I will look into these later on once I see the freaking green test.
Running on public networks is also a pain in the ass for testing. There is this thing I could try https://github.com/blockfrost/blockfrost-backend-ryo but it would require a lot of fiddling around and I need a solution I can ship with hydra for testing purposes.
Another idea I just got is why don't I run some hydra-chain-observer test using the code from blockfrost observations? It might be faster for me to figure out where the error is? Argh, no extensive testing is in there, just a bit of observation testing.
I noticed that slots seems to be off when following the chain:

  new block hash: BlockHash "a014462e775f5df1a2830a175b06b1b4d98e0fa89999622eee32a74bc3a64c2e"
new slot number: 80051427
new block hash: BlockHash "c08a8b97f9d6389913ec69c5634d33b903e8b241bac5e1e54035bc7dde7227b1"
new slot number: 80051427

I see in the blockfrost Block that there is a _blockSlot and _blockEpochSlot. Not sure what is the difference, I'll try using _blockEpochSlot to see how it looks.
Ah no, this is wrong. Let's compare the block hash vs. slot number on cardano explorer and blockfrost. Btw SlotNo on cardano is Word64 while blockfrost uses Integer
Maybe this was just me being tired, block slots seem fine. Perhaps I need to somehow implement onRollBackward?
I added 5 block confirmations instead of 1 - right now I am very tired and it doesn't seem I will find the problem like this. A small break is in order. Argh, more confirmations require more waiting on the UTxO.
Before that I wanted to try filtering the UTxO I get from queryUTxOForTxIn
- for some reason this doesn't seem to work - I get BadInputsUTxO error. Need to raise this to blockfrost people.
I will leave this for now just to refresh my brain and instead focus on making my silly typeclass a bit better.

2025-05-07

SB on hydra-node using blockfrost

I managed to finally get to a working tx submission by using the CostModelsRaw from blockfrost. Plain CostModels contain the field names and they are not in order I was expecting them to be in. Now we can finally evaluate txs from the wallet.
After this fix I noticed I am missing the whole part of code to actually submit transactions in the Direct module.
I copy/pasted relevant code from cardano implementation that worked out of the box.
Now I had to bump the time it takes to observe some tx in the DirectChainSpec since blockfrost takes a bit of time more.
And now I see BadInputs again after observing the initialization/or sometimes we fail to observe initialization.
Pretty sure the code I copy/pasted with some changes is causing this - this should be the final step before blockfrost integration should start working.
Failure BadInputsUTxO happens when trying to incrementally commit some UTxO. I'll render the tx to see how it looks like first.
By looking at the tx it is obvious that seedFromFaucetBlockfrost needs await for faucet utxo but it needs to return whatever utxo we seeded to some actor.
For some reason just awaiting with the recipient address hangs - I would expect to be able to see the ouput with corresponding input txid working. Perhaps there is another bug related to how we construct cardano utxo (there is a related TODO about the two almost identical ways of doing it).

2025-05-05

SB on hydra-node using blockfrost

Let's see if publishing of scripts still works and start from there. I should inspect how EraHistory is assembled since that is related for tx evaluation which is where the error comes from.
Problem is that tx evaluation thinks PlutusV3 is not enabled yet and that information comes from looking at EraHistory.
There is still a problem with flaky waiting on UTxO from blockfrost which sometimes results in BadInput error, so I might also choose to fix that problem first.
Script publishing (with waiting) still works, good.
Let's focus first on BadInput error. This is how the current wallet UTxO looks like:

ubuntu@ip-10-1-6-128:~$ sudo ./cardano-cli-x86_64-linux query utxo --socket-path preview/node.socket --address addr_test1vztc80na8320zymhjekl40yjsnxkcvhu58x59mc2fuwvgkc332vxv --testnet-magic 2
                           TxHash                                 TxIx        Amount
--------------------------------------------------------------------------------------
dcec59151ed3616b0836bdb5d3e77f99cbe8525c83683030f1868cf4b2108ad3     1        143491445598 lovelace + 1 13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3.487964726120446f6f6d202d2033726420506c6163652054726f706879 + 1 19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07.487964726120446f6f6d202d20326e6420506c6163652054726f706879 + 1 1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11.2331 + 1 6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a.2331 + 1 cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b.2331 + 69918000000 d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4.0014df105553444d + 1 dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee.487964726120446f6f6d202d2034746820506c6163652054726f706879 + TxOutDatumNone

So more than enough ada but also some tokens all in one output. Why would this be problematic?
Running the test I get:

       FaucetBlockfrostError {blockFrostError = "BlockfrostBadRequest \"{\\\"contents\\\":{\\\"contents\\\":{\\\"contents\\\":{\\\"era\\\":\\\"ShelleyBasedEraConway\\\",\\\"error\\\":[\\\"ConwayUtxowFailure (UtxoFailure (ValueNotConservedUTxO (MaryValue (Coin 0) (MultiAsset (fromList []))) (MaryValue (Coin 143491445598) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash \\\\\\\"13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3\\\\\\\"},fromList [(\\\\\\\"487964726120446f6f6d202d2033726420506c6163652054726f706879\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07\\\\\\\"},fromList [(\\\\\\\"487964726120446f6f6d202d20326e6420506c6163652054726f706879\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11\\\\\\\"},fromList [(\\\\\\\"2331\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828\\\\\\\"},fromList [(\\\\\\\"487964726120446f6f6d202d2031737420506c6163652054726f706879\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9\\\\\\\"},fromList [(\\\\\\\"487964726120446f6f6d202d2031737420506c6163652054726f706879\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c\\\\\\\"},fromList [(\\\\\\\"487964726120446f6f6d202d2031737420506c6163652054726f706879\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a\\\\\\\"},fromList [(\\\\\\\"2331\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e\\\\\\\"},fromList [(\\\\\\\"487964726120446f6f6d202d2031737420506c6163652054726f706879\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b\\\\\\\"},fromList [(\\\\\\\"2331\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4\\\\\\\"},fromList [(\\\\\\\"0014df105553444d\\\\\\\",69918000000)]),(PolicyID {policyID = ScriptHash \\\\\\\"dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df\\\\\\\"},fromList [(\\\\\\\"487964726120446f6f6d202d2031737420506c6163652054726f706879\\\\\\\",1)]),(PolicyID {policyID = ScriptHash \\\\\\\"fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee\\\\\\\"},fromList [(\\\\\\\"487964726120446f6f6d202d2034746820506c6163652054726f706879\\\\\\\",1)])])))))\\\",\\\"ConwayUtxowFailure (UtxoFailure (BadInputsUTxO (fromList [TxIn (TxId {unTxId = SafeHash \\\\\\\"dcec59151ed3616b0836bdb5d3e77f99cbe8525c83683030f1868cf4b2108ad3\\\\\\\"}) (TxIx {unTxIx = 1})])))\\\"],\\\"kind\\\":\\\"ShelleyTxValidationError\\\"},\\\"tag\\\":\\\"TxValidationErrorInCardanoMode\\\"},\\\"tag\\\":\\\"TxCmdTxSubmitValidationError\\\"},\\\"tag\\\":\\\"TxSubmitFail\\\"}\""}

So dcec59151ed3616b0836bdb5d3e77f99cbe8525c83683030f1868cf4b2108ad3#1 seems invalid? Why? This is exactly the single UTxO I would like to be able to spend.
This is the same tx in the explorer https://preview.cexplorer.io/tx/dcec59151ed3616b0836bdb5d3e77f99cbe8525c83683030f1868cf4b2108ad3 and it seems like after this tx we have the following UTxO https://preview.cexplorer.io/address/addr_test1vztc80na8320zymhjekl40yjsnxkcvhu58x59mc2fuwvgkc332vxv/tx#data
It seems like we return wrong next UTxO from publishing scripts since I see that the last UTxO I spent was the one seeding 100 ada. Then we ran the test which published the scripts but then seeding first time worked and again failed after the initial 100 ada seed.
Both scripts publishing and seeding from UTxO call the same awaitTransactionId function. Altering the script publishing to await after every tx results in the same error.
Perhaps I could try just seeding three UTxO and print out the actuall utxo so I can try to pinpoint where the error comes from.
I see in the explorer that two transactions worked but the error I see is related to the first UTxO. How can it be that the two txs succeeded but the error I see comes from the UTxO I saw after the first tx? Third seeding can't possibly take in as the input some UTxO which is the output of a first tx?
I noticed that await uses queryUTxOByTxIn while getting the faucet utxo uses queryUTxO. There could be a problem where queryUTxO reports the wrong utxo. There is also toCardanoUTxO and it's prime version that need to be unified.
In general queryUTxO result can't be trusted. Seems like we get better UTxO information when we query transaction UTxO. We could try to chain these seed operations so if we provide Nothing to the seeding function it tries to query the UTxO - otherwise we re-use the UTxO we already know.
This could work but it is not exactly what I want since for example if you publish the scripts first (or do any other information) how do you know that after that queried UTxO will be the last one really?
In general we need a secure way to know this is the latest UTxO of specific address.
After chaining of the txs I get:


       FaucetBlockfrostError {blockFrostError = "BlockfrostBadRequest \"{\\\"contents\\\":{\\\"contents\\\":{\\\"contents\\\":{\\\"era\\\":\\\"ShelleyBasedEraConway\\\",\\\"error\\\":[\\\"ConwayUtxowFailure (MissingVKeyWitnessesUTXOW (fromList [KeyHash {unKeyHash = \\\\\\\"f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d\\\\\\\"}]))\\\"],\\\"kind\\\":\\\"ShelleyTxValidationError\\\"},\\\"tag\\\":\\\"TxValidationErrorInCardanoMode\\\"},\\\"tag\\\":\\\"TxCmdTxSubmitValidationError\\\"},\\\"tag\\\":\\\"TxSubmitFail\\\"}\""}

So we are missing a sig somewhere. This happens after the initial 100 ADA seed.
What I want is to be able to query the UTxO and know it is the actuall updated UTxO after the last sent tx. I could provide Tx hash I expect to see in the utxo and filter like that.
Let's do one round of tracing to try to spot where the problem is. We start with faucet utxo "6c2fcbb6b7d79a08eee4588feabeb79e2d8b3d77d4c916291c39ba514bb449f8#1". From the script publishing we see:

303f49141d3042bc0c25105ab0686ca274e92aa23ac83a65b260b98961c2108b#0
https://preview.cexplorer.io/tx/303f49141d3042bc0c25105ab0686ca274e92aa23ac83a65b260b98961c2108b

9d86e58a5a9afb24bfd22dfef0f171cb36496e7c510ac784b948259c4c4fd5ac#0
https://preview.cexplorer.io/tx/9d86e58a5a9afb24bfd22dfef0f171cb36496e7c510ac784b948259c4c4fd5ac

8a0243a3ab24d0acd9d64111f6a826485173236c36a77c54a6b53d425b1e81c7#0
https://preview.cexplorer.io/tx/8a0243a3ab24d0acd9d64111f6a826485173236c36a77c54a6b53d425b1e81c7

When we try to find faucet utxo after that we get 6c2fcbb6b7d79a08eee4588feabeb79e2d8b3d77d4c916291c39ba514bb449f8#1 which is the utxo we got BEFORE publishing of scripts. So this means either the await in the script publishing is not doing it's job or two different queries to get the latest blockfrost utxo are behaving different.
I tried using new function awaitUTxO that also tries to wait for specific utxo with txid but it seems that it hangs forever. Need to fix that one first.
Ok, the function seems to be working - it was just a matter of waiting on the last tx id from publishing txs instead of trying to find all three (which makes sense, I was just stupid).
In general is seems like blockfrost api is behaving different when you have a tx hash and want to get the UTxO for it, and, in contrast, when you have the address and just want to get the UTxO for that address.
One problem solved all though I think heavy refactoring should be next in line but let's focus on another problem at hand first.
I am not able to post an init tx because of: LedgerLanguageNotAvailableError
This should be tightly related to how we construct EpochInfo (and tightly related EraHistory) so I need to revisit this and compare what we get from a real cardano-node and blockfrost api.
I noticed that the filtering was reversed! This should fix the last problem but I still see ocasinal BadInputs and this is because we can't know what is the latest fresh faucet UTxO on each test run! This is so annoying - the endpoint that gives you utxo for address is not updated immediately so if you don't know the exact txid to await for then we need to wait a bit after each test run in order to hope that the UTxO we see after hydra scripts publishing is correct (or perhaps even the script publishing was failing because of the same problem).
I noticed that major and minor version in ProtVer are reversed 😱 This must be the problem I am facing.
Ha! found it. Now I get the error related to overspending the budget which I need to look into next.
I noticed that memory and steps in MaxTxExecutionUnits is reversed! Another found error! BUT I still see overspent budget.

April 2025

2025-04-30

SN on fixing deposits

Last step to make the node safe for deadlines too soon is to fix it not accepting snapshots if deadline too soon.
This is logic that needs to go into the ReqSn handling.
I wondered why does the ReqSn contain UTxO if the node needs to know about the deposit anyways? Make this a TxId too like the normal requested transaction ids to confirm.
As I added a created :: SlotNo to the DepositObservation, I realized this is the first of its kind where we want to observe a slot from the chain while we actually need its wall clock equivalent in the HeadLogic. Where to convert?
Problem: convertObservation is also used by the hydra-chain-observer and we can only convert time in a hydra-node (we only need it converted in the node).
- Could: front load conversion of slot to time to deposit creation, put it into the datum and only observe if it matches?
- Does not make sense.. we could convert on observation right away..
- Providing a TimeHandle to convertObservation makes sense if we can just switch the hydra-chain-observer to use the HeadObservation type directly.
Finally, to make the deposit only be signed after becoming active (settled), I reached for the Wait outcome to have the snapshot request be handled later, once the deposit is active. However, this requires significant increase in TTL and wait between re-enqueuing to ensure we can wait long enough for the deposit to become active (> deposit period).
Increasing the TTL is likely not a good idea because the input queue is not persisted (right now?). An alternative could be to shift the burden onto the snapshot leader and have them re-submit snapshot requests until the head moves forward again… this sounds actually quite nice as it would make the overall protocol more robust?

SB on hydra-node using blockfrost

Continuing from yesterday I need to make blockfrost awaiting work. The UTxO after publishing scripts is not good and I have duplicated logic for when you get script outputs and regular pub key outputs I need to unify.
After unifying these two functions I see that await works but sometimes it also gives back the wrong utxo which causes next tx in line to fail with BadInputs.
This is a bit weird since it seems api calls to blockfrost are somehow flaky?
Even after publishing scripts and seeding from a faucet works - when assembling a InitTx in wallet I get this dreaded error:

       ScriptFailedInWallet {redeemerPtr = "ConwayMinting (AsIx {unAsIx = 0})", failureReason = "ValidationFailure (WrapExUnits {unWrapExUnits = ExUnits' {exUnitsMem' = 0, exUnitsSteps' = 0}}) (CodecError (LedgerLanguageNotAvailableError {sdeAffectedLang = PlutusV3, sdeIntroPv = 9, sdeThisPv = 0})) [] (PlutusWithContext {pwcProtocolVersion = Version 0, pwcScript = Left (Plutus {plutusBinary = \"WRWiAQEAMzMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyIiIykwAQApEUgAyJQGpLJksmRkZGSqZmrmgARGAEImSqZmrmgARGAAImBkBiZuHSACADM3DpAAABGqueACNVc6ACbqgBImYESSAQNNMDYAWTMwICIyMzVzQAMACAEZuPACABMCEAEwI5ABkRABpCZgXkSyADGABIhkAKRLJkZmrmgAYAEAIzcQACkABEwBgA4wAgADGAEiZmZkZEREplJmBSkhA00wMQAyMwNiJZABjACkQrKzIBEikASRGRmauaABgAQAjNx4AgARGQCJFIAkiMjM1c0ADAAgBGbhwAwASMAJGABEwBAAQAEzA1IlkAGMACRDADkAKRABJgCAAkqADIiIiIiIiIiIBBMjIyMjIVkzAuSRA00wMgAyMzVzQAMACAEZuHMlkzMC0iMjM1c0ADAAgBGbjwAgAVAGMDBQB5CYAQAMUgABkZkRkRgBAAmB6RLIAMQA5EMgBSIzAIABMAYAMkAGRABJABkQAKACRGbgAAgAYG4zcAoAKQAUVkzAuSQBA00wMwAzIyIwAgATA7IlkAGMAKRDIAUiMlUzNXNAAiJgZgBCJgDgCGbjwAgBxQBZUAORAAorKyZgXJIBA00wNAAyMzVzQAMACAEZuHUAEzALESIyIwAgATA8IlkAGIAciEzAGACMAQAEoAkTMDMlkzMC0iMjM1c0ADAAgBGbjwAgAVAGABkKyADEwO0kAQNNMDgAkQyAFIlkAOKyZGZq5oAGABACM3DgApABRgARMD9JAQNNMDgASITBBSQBA00wOAARMDpJAQNNMDcAKAJGAEismYGZLIAMVAxkKyagbAAyEzAxSQEDTTEyADIzNXNAAwAIARm48AFQCImB2kkDTTEyAEhUDIZgdESyADGABIhgByAFIgAkwBAAQzAMAKUAaJmBckkDTTA1AFkyMzVzQAMACAEZuPlQApEQAZQBYmSzIAMikAuRGRmauaABgAQAjNx4AgARGQAZFIBciMjM1c0ADAAgBGbhwAwASMAJKgBSIgAkYASMAJGAEjACRgBBlQAZEQAIVmVABkQASKgYSFZMzMzA0MjIyMiIyUyUzNXNABCKyZmB7ACIoAMAEAKACAByGQAZEsmAUAFIVkzMEGAERQAYAIAUAEACkMgAyJZMwDADQApCsmZgiwAiKADABACgAgAUhkAGRLJqCMAFIVkzMEmAERQAQAgAUhWTMzMwSSIyVTM1c0ACIrJmYJsAIigAwAQAoAIAFIZABkSyagnABSFZMzBRgBEUAEAIAFIVk1AdABkMAGACAHABIwAkYASMAIjACCMAIzcOkAAAEUAIoARQAigBAAyGADAFALgDwA4AJEwDkmRMA1JkTAMSYiYBKTImAQkxEwBUmRMARJiJgApMETABSYyIyVTM1c0ACIrJmYH8AIigAgBABZCsmZmZgfkRkqmZq5oAERWTMwQ4ARFABgAgBQAQAKQyADIlk1BEACkKyZmCPACIoAMAEAKACABSGQAZEsmYCQCYAUhWTMwS4ARFABgAgBQAQAKQyADIlkwGAApCsmZgnwAiKADABACgAgAUhkAGRLJqA6AFIVkzMFOAERQAQAgAUhWTUFIAGQwAYAIB8AuAPADgAkYASMAJGAERgBIwAiMAJGAERgBIwAiMAIIwAjNw6QAAARQAigBFACKAEADIYAMAcAEiYAiTImAGkwRMANJjNw6QAQApkRkqmZq5oAERWTMwP4ARFABACACyFZMzMzA/IjJVMzVzQAIismZghwAiKADABACgAgAUhkAGRLJqCIAFIVkzMEeAERQAYAIAUAEACkMgAyJZMwEgEwApCsmZglwAiKADABACgAgAUhkAGRLJgMABSFZMzBPgBEUAGACAFABAApDIAMiWTUB0AKQrJmYKcAIigAwAQAoAIAFIZABkSyagQgBSFZMzBXgBEUAGACAFABAApDIAMiWTUFgAKQrJmYLcAIigAwAQAoAIAFIZABkSyaguABSFZMzBfgBEUAGACAFABAApDIAMiWTUGAAKQrJmYMcAIigAwAQAoAIAFIZABkSyZgXKDIAFIVkzMGeAERQAQAgAUhWTUDMAGQwAYAIEcB+A3AXgJwD4BcAeAHABIwAkYASMAIjACRgBEYASMAIjACRgBEYASMAIjACRgBEYASMAIjACRgBEYASMAIjACCMAIzcOkAAAEUAIoARQAigBAAyGADABABImAIkyJgBpMETADSYzcOkAIAKRkqmZq5oAERgAwAgjACM3DpADACGbh0gAAAiMjMzMDkigBFACABKAEUAJTMD0iEzA7gAwABEsmAKAFIVk1MAUSABACkMAGAGAFABIwAkTKAEamAIJAAgAhIAElk1A2ABkMAEAFGAEIzMzMDYiMlUzNXNAAiKyZmB1ACIoAIAQAKQrJqAMADIYAIAKMAJGAEEYARm4dIAAAIoARQAigBFACABEzMzA0IoARQAigBFABABFACKAEUAIoARQAgAZCsgAyFQMoqBjIiIYAIAUAGZgHiJEZEYAQAJggESyADEAORCZgDABGAIACADSFQMkTA4SQQNNMDkASFQMQrJmASAMoAcTA2SRA00xMQCRCsgAxACkQmB0kgEDTTExAATMDYiWQAYwAJEMAOQApEACmAIACGYBAAygBCYFAAUYAQZABkRABiM1UAEiM3AAApABJAAERmBgRLIAMYAEiEyWZADkRFIAkikAKUAKRkqmZq5oAERgAwAQBoA4IwAjNx4AIBwkMAMAGAEiACMAQAFIAMiIiIiIiIiIgDgBQBIBAAIwBQBopNFQGUiUBqSgMyJQGpKAyEZGRkZKpmauaABETKACZGRkZKpmauaABETIyMjIyMjIyMjIyMjIygAmRqA8brAATV0ICEyNQHzdYACauhAOmRqBCbrAATV0IBs3WmroQDJmYErrjMCV1zrTV0IBcyNQIzdYACauhAKmZgSgUOtNXQgEzNTIyEiMjIyMlUzNXNAAiMAEyMjIyVTM1c0ACIwATMAwAo1dCAFMAs1dCauiACCJglglGbh0gAAAjVXPABGqudABN1Rq6EAKZGRkZKpmauaABEYAJmAYAUauhACmAWauhNXRABBEwSwSjNw6QAAARqrngAjVXOgAm6o1dCauiACCJgjgjGbh0gAAAjVXPABGqudABN1QAJGRkZGSqZmrmgARGAEImSqZmrmgARGAAImCMCKZuHSACADM3DpAAABGqueACNVc6ACbqgASIyMjIyVTM1c0ACIwAhEyVTM1c0ACIwATAHNXQgBhEyVTM1c0ACIwBBEwRwRjNw6QAgAhm4dIAIAMzcOkAAAEaq54AI1VzoAJuqABdaauhAImYGDrjV0IA8zMCUjIyMjJVMzVzQAIjADN1xq6EAIImSqZmrmgARGASYF5q6EAMImSqZmrmgARGAOYF5q6EAQImSqZmrmgARGACbrTV0IAswLjV0Jq6IAUImSqZmrmgARGAWYGBq6EAYImSqZmrmgARGAKbrTV0IA8wLjV0Jq6IAcImCSCQZuHSAKAHM3DpAEADGbh0gBgBTNw6QAgAhm4dIAIAMzcOkAAAEaq54AI1VzoAJuqABAtNXQgDTMwJXXAWmroQBZuuNXQgCTMwJQJzMCUC8jIyMjJVMzVzQAIjACETJVMzVzQAIjAEETJVMzVzQAIjAAETBGBFM3DpACACGbh0gAgAzNw6QAAARqrngAjVXOgAm6oAE1dCAHMjUCQ3WAAmroQApmBc601dCADMwLnWmroTV0QAI1dEACauiABNXRAAmrogATV0QAJq6IAE1dEACauiABNXRAAmrogATV0QAJq6IAE1dEACauiACETAxAwM3DpAAABGqueACNVc6ACbqjV0IAc1dCADMjIyMlUzNXNAAiMAM3XGroQAgiZKpmauaABEYBJgOGroQA5mBCBAauhNXRABhEyVTM1c0ACIwBzAcNXQgCBEyVTM1c0ACIwATdaauhAFmA2auhNXRAChEyVTM1c0ACIwCzAdNXQgDBEyVTM1c0ACIwBTdaauhAHmA2auhNXRADhEwNgNTNw6QBQA5m4dIAgAYzcOkAMAKZuHSAEAEM3DpABABmbh0gAAAjVXPABGqudABN1Rq6E1dEACNXRABCJgWAVmbh0gAAAjVXPABGqudABN1QAJmBGRCZgQwACKAGYAwAU1MAQSABABCQAJmBERCZgQQACKAGYAoAU1MAQSABABCQAJGRkZGSqZmrmgARGACYB5q6EAKYA5q6E1dEAEETAlAkM3DpAAABGqueACNVc6ACbqgATMCAiEzAegAEUAMwBQApqYAgkACACEgASMjIyMlUzNXNAAiJkZQATIyMjJVMzVzQAIjABMBI1dCAFMwGCMjIyMlUzNXNAAiMAEwFzV0IAQRMlUzNXNAAiJlADN1pq6EASbrTV0IAM3WmroTV0QAI1dEAGImBeBcZuHSACADM3DpAAABGqueACNVc6ACbqgATV0Jq6IAIImBSBQZuHSAAACNVc8AEaq50AE3VGroQBJmYBbrjMAt1zrTV0IAUyMjIyVTM1c0ACIwABEyVTM1c0ACIwBTdcauhADCJkqmZq5oAERgBmroQBAiYFYFRm4dIAQAQzcOkAEAGZuHSAAACNVc8AEaq50AE3VGroQAZmAo641dCauiABGrogATV0QAQiYEYERm4dIAAAI1VzwARqrnQATdUACZgPEQmYDkAAigBmAQAFNTAEEgAQAQkACZgOkQmYDcAAigBmAKAFNTAEEgAQAQkACRkZGRkqmZq5oAERMoAJutNXQgBzAKNXQgAzIyMjJVMzVzQAIiZQCTMBUBY1dCAHNXQgAzMBV1xq6E1dEACNXRABCJkqmZq5oAERgAmYCoCxq6EAOZGRkZKpmauaABEYAJutNXQgBTdaauhNXRABBEwKgKTNw6QAAARqrngAjVXOgAm6o1dCauiADCJkqmZq5oAERgFmZgGgIOtNXQgCTMBZ1xq6E1dEAIETJVMzVzQAIjAHMwFwGDV0IAoRMlUzNXNAAiJkZQDTMBoBs1dCARMwHAFDV0IAUzMBEBR1pq6EAHJkZGRkqmZq5oAERgAm601dCAFN1pq6E1dEAEETAvAuM3DpAAABGqueACNVc6ACbqjV0Jq6IAGRGYDAAQAIauiABNXRADCJkqmZq5oAERgCmYDIDRq6EAeZGRkZKpmauaABETMB11xq6EAIRMC4C0zcOkAAAEaq54AI1VzoAJuqNXQmrogBwiZKpmauaABEYAQiYFYFRm4dIAwAgzcOkAUAOZuHSAIAGM3DpADACmbh0gBABDNw6QAQAZm4dIAAAI1VzwARqrnQATdUauhNXRAAjV0QAQiYEAD5m4dIAAAI1VzwARqrnQATdUACRGRGoARurABMwHSITMBuAARQA4AJgDGqudACmAKaq54AJNTAEEgAQAQkACZEZGRkZKpmauaABEYBpgEGroQApmAc601dCauiACCJkqmZq5oAERgJmASauhADmYB7rTV0Jq6IAMImSqZmrmgARGAGYBRq6EASYBBq6E1dEAIETJVMzVzQAIiZQCzAMNXQgDTAKNXQgAzdaauhNXRAAjV0QAoiZKpmauaABEYBJgGGroQBputNXQmrogBgiZKpmauaABEYCpgGmroQBwiZKpmauaABEYCJgHGroQCJutNXQmrogCAiZKpmauaABEYApuuNXQgEzdcauhNXRAEhEyVTM1c0ACIwBzdcauhAKm601dCauiAKCJkqmZq5oAERgAmAiauhALmAiauhNXRAFhEyVTM1c0ACIwDzASNXQgGBEwKQKDNw6QCgBhm4dIBIAszcOkAgAUZuHSAOAJM3DpAGAEGbh0gCgBzNw6QBAAxm4dIAYAUzcOkAIAIZuHSACADM3DpAAABGqueACNVc6ACbqgATIjIyMjJVMzVzQAIjABN1xq6EAIImSqZmrmgARGAKYA5q6EAMImSqZmrmgARGAGbrjV0IAkwCDV0Jq6IAQImBCBAZuHSAEAEM3DpABABmbh0gAAAjVXPABGqudABN1QAJGRkZGSqZmrmgARGACYA5q6EAIImSqZmrmgARGAEImSqZmrmgARGAIImBAA+ZuHSAEAEM3DpABABmbh0gAAAjVXPABGqudABN1QAJGRkZGSqZmrmgARGACYAxq6EAIImSqZmrmgARGAGYA5q6EAMImSqZmrmgARGAKbrjV0IAgRMB8B4zcOkAIAIZuHSACADM3DpAAABGqueACNVc6ACbqgASMjIyMlUzNXNAAiMAE3XGroQAgiZKpmauaABEYAZuuNXQgBhEwHQHDNw6QAQAZm4dIAAAI1VzwARqrnQATdUACRkZGRkqmZq5oAERgAm641dCAFN1pq6E1dEAEETAbAaM3DpAAABGqueACNVc6ACbqgATAWIiMlUzNXNAAiJgMpIBA1BUMwARMlUzNXNAAiJmAKZuBAYAEM3AgMABiJlABM3CACgAzNwgAgAIzAGAEADM3EABALmbhwAQFjAVIiMlUzNXNAAiIAYiZgCABGbhgAwAjNw4AIComAmkgQNQVDUAIAEiMjIyMlUzNXNAAiMAIRMlUzNXNAAiMAEwBzV0IAYRMBgBczcOkAAAGZuHSACACNVc8AEaq50AE3VAAkZGRkZKpmauaABEYAJuuNXQgBTdaauhNXRABBEwFQFDNw6QAAARqrngAjVXOgAm6oAEjIjUAI3WAAmYCJEJmAfAAIoAZgCgBTUwBBIAEAEJAAmAeRLIAMYAUiGQApEZKpmauaABETIzNXNAAwAIARm4cAJIAIRMAcAQzceAEkULSHlkcmFIZWFkVjEACIyIwAgATAQIlkAGMAKRDIAUiWTMAgAIAeMAEAFEwBgAwZFIAMiKQAZEoA8gAZEoA8lAGkSgDyUAYEyYzVziSQJMaACAARLIAMYAETNXOABQAhSADIiIiIiIiIiIAwJgEpIBA00xMgATAISRA00wOQATAHSRA00xMAAjIjACABMAgiWQAYwAJEKyYAoAUTAEABjACBMzMwASKAEUAIoARQAigAgAiIiIlMzMzV0gAImRmAOaq50AE1VzwAJuqABEwBTdWACJgCG6wAETADN1oAImAEbrgASIlMzVXPgAiAGJmAEauhABNXRAApAAJIEDUFQxACMmM1c4ADAAIyMAEAEjACIzACACABSJHMihAaXIrEgWsNzrWc4x/CJY44fego8Clh0vIEUASIEcDjURWix8E8aOzY105Jh8BNRTnjN2Q74guzJ0vQAzM1EiACIoAIAUAEQlIAUAEkUgDFnmis0iW8eo2Ro3sYNEBZlBGtKsH/zhQPkoxLw1HU0ASAAB\"}), pwcScriptHash = ScriptHash \"c48ad46525f11f52e9f67ef942a3e6e727966b89c43e7c323de5ad55\", pwcArgs = ScriptContext {scriptContextTxInfo = TxInfo {txInfoInputs = [TxInInfo {txInInfoOutRef = TxOutRef {txOutRefId = 0c59e68acd225bc7a8d91a37b183440599411ad2ac1ffce140f928c4bc351d4d, txOutRefIdx = 0}, txInInfoResolved = TxOut {txOutAddress = Address {addressCredential = PubKeyCredential f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d, addressStakingCredential = Nothing}, txOutValue = Value {getValue = Map {unMap = [(,Map {unMap = [(\"\",100000000)]})]}}, txOutDatum = NoOutputDatum, txOutReferenceScript = Nothing}}], txInfoReferenceInputs = [], txInfoOutputs = [TxOut {txOutAddress = Address {addressCredential = ScriptCredential 0e35115a2c7c13c68ecd8d74e4987c04d4539e337643be20bb3274bd, addressStakingCredential = Nothing}, txOutValue = Value {getValue = Map {unMap = [(,Map {unMap = [(\"\",0)]}),(c48ad46525f11f52e9f67ef942a3e6e727966b89c43e7c323de5ad55,Map {unMap = [(\"HydraHeadV1\",1)]})]}}, txOutDatum = OutputDatum (Datum {getDatum = Constr 0 [Constr 0 [I 10000],List [B \"\\213\\191J?\\204\\231\\ETB\\176\\&8\\139\\204'I\\235\\193H\\173\\153i\\178?E\\238\\ESC`_\\213\\135xWj\\196\"],B \"\\196\\138\\212e%\\241\\USR\\233\\246~\\249B\\163\\230\\231'\\150k\\137\\196>|2=\\229\\173U\",Constr 0 [B \"\\fY\\230\\138\\205\\\"[\\199\\168\\217\\SUB7\\177\\131D\\ENQ\\153A\\SUB\\210\\172\\US\\252\\225@\\249(\\196\\188\\&5\\GSM\",I 0]]}), txOutReferenceScript = Nothing},TxOut {txOutAddress = Address {addressCredential = ScriptCredential c8a101a5c8ac4816b0dceb59ce31fc2258e387de828f02961d2f2045, addressStakingCredential = Nothing}, txOutValue = Value {getValue = Map {unMap = [(,Map {unMap = [(\"\",0)]}),(c48ad46525f11f52e9f67ef942a3e6e727966b89c43e7c323de5ad55,Map {unMap = [(0xf8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d,1)]})]}}, txOutDatum = OutputDatum (Datum {getDatum = B \"\\196\\138\\212e%\\241\\USR\\233\\246~\\249B\\163\\230\\231'\\150k\\137\\196>|2=\\229\\173U\"}), txOutReferenceScript = Nothing}], txInfoFee = 0, txInfoMint = UnsafeMintValue (Map {unMap = [(c48ad46525f11f52e9f67ef942a3e6e727966b89c43e7c323de5ad55,Map {unMap = [(\"HydraHeadV1\",1),(0xf8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d,1)]})]}), txInfoTxCerts = [], txInfoWdrl = Map {unMap = []}, txInfoValidRange = Interval {ivFrom = LowerBound NegInf True, ivTo = UpperBound PosInf True}, txInfoSignatories = [], txInfoRedeemers = Map {unMap = [(Minting c48ad46525f11f52e9f67ef942a3e6e727966b89c43e7c323de5ad55,Redeemer {getRedeemer = Constr 0 []})]}, txInfoData = Map {unMap = []}, txInfoId = 71ebd1ece4ecc2001ca38f6d5a56b182f9103c58bfb3e3afa12c24f8308b523c, txInfoVotes = Map {unMap = []}, txInfoProposalProcedures = [], txInfoCurrentTreasuryAmount = Nothing, txInfoTreasuryDonation = Nothing}, scriptContextRedeemer = Redeemer {getRedeemer = Constr 0 []}, scriptContextScriptInfo = MintingScript c48ad46525f11f52e9f67ef942a3e6e727966b89c43e7c323de5ad55}, pwcExUnits = WrapExUnits {unWrapExUnits = ExUnits' {exUnitsMem' = 10000000000, exUnitsSteps' = 14000000}}, pwcCostModel = CostModel PlutusV3 [100788,420,1,1,100181,726,719,0,1,1000,173,0,1,1000,59957,4,1,11183,32,207616,8310,4,201305,8356,4,962335,18,2780678,6,442008,1,52538055,3756,18,267929,18,76433006,8868,18,52948122,18,1995836,36,3227919,12,901022,1,166917843,4307,36,284546,36,158221314,26549,36,74698472,36,333849714,1,254006273,72,2174038,72,1006041,43623,251,0,1,16000,100,16000,100,16000,100,16000,100,16000,100,16000,100,16000,100,16000,100,100,100,16000,100,94375,32,132994,32,61462,4,107878,680,0,1,72010,178,0,1,22151,32,107490,3298,1,91189,769,4,2,85848,123203,7305,-900,1716,549,57,85848,0,1,1,1000,42921,4,2,24548,29498,38,1,898148,27279,1,51775,558,1,39184,1000,60594,1,106057,655,1,141895,32,83150,32,15299,32,76049,1,13169,4,1293828,28716,63,0,1,2261318,64571,4,22100,10,28999,74,1,28999,74,1,43285,552,1,44749,541,1,33852,32,68246,32,72362,32,7243,32,7391,32,11546,32,85848,123203,7305,-900,1716,549,57,85848,0,1,90434,519,0,1,74433,32,100181,726,719,0,1,85848,123203,7305,-900,1716,549,57,85848,0,1,95336,1,1,85848,123203,7305,-900,1716,549,57,85848,0,1,180194,159,1,1,1964219,24520,3,159378,8813,0,1,955506,213312,0,2,270652,22588,4,1457325,64566,4,158519,8942,0,1,20467,1,4,0,141992,32,100788,420,1,1,81663,32,59498,32,20142,32,24588,32,20744,32,25933,32,24623,32,43053543,10,53384111,14333,10,43574283,26308,10,281145,18848,0,1,100181,726,719,0,1]})"}

Seems like this is happening when evaluating scripts and for some reason intro protocol version is 9 (PlutusV3) but current protocol version is zero.
evalTxExUnits is relevant here since it is responsible for throwing this error. There is also evalTxExUnitsWithLogs which should give more context.
I think I should focus first on getting the blockfrost utxo always to be bulletproof before I go further into wallet errors.
Currently there is only one UTxO at the faucet address:

                            TxHash                                 TxIx        Amount
--------------------------------------------------------------------------------------
f9d95b0f651d772ad4a98d4855d7f7f2ad259f8946e2145bafe96c97b45a8144     1        133920252523 lovelace + 1 13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3.487964726120446f6f6d202d2033726420506c6163652054726f706879 + 1 19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07.487964726120446f6f6d202d20326e6420506c6163652054726f706879 + 1 1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11.2331 + 1 6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a.2331 + 1 cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b.2331 + 69918000000 d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4.0014df105553444d + 1 dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee.487964726120446f6f6d202d2034746820506c6163652054726f706879 + TxOutDatumNone

and now I get constant failure with BadInput error! Clearly blockfrost doesn't like this utxo at all. Let me see if I seed from the faucet again and have two UTxO if the test will work always.

Oh look at this one:

                           TxHash                                 TxIx        Amount
--------------------------------------------------------------------------------------
642d74562744eb5739f36845be0a6462c34496c758a58b0e3a91ec0d8eb70ffb     0        10000000000 lovelace + TxOutDatumNone
98a404a9e0d0fceec5c3c0d9ea45802df09db9f5c0b8f98dfbefa2b3da7e3b9f     1        133838102650 lovelace + 1 13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3.487964726120446f6f6d202d2033726420506c6163652054726f706879 + 1 19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07.487964726120446f6f6d202d20326e6420506c6163652054726f706879 + 1 1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11.2331 + 1 6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a.2331 + 1 cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b.2331 + 69918000000 d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4.0014df105553444d + 1 dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee.487964726120446f6f6d202d2034746820506c6163652054726f706879 + TxOutDatumNone
ubuntu@ip-10-1-6-128:~$ sudo ./cardano-cli-x86_64-linux query utxo --socket-path preview/node.socket --address addr_test1vztc80na8320zymhjekl40yjsnxkcvhu58x59mc2fuwvgkc332vxv --testnet-magic 2
                           TxHash                                 TxIx        Amount
--------------------------------------------------------------------------------------
25727480009bb6d3b560914d821da88304ccec98a7439a694ce5ed53bbabab6e     1        143673796876 lovelace + 1 13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3.487964726120446f6f6d202d2033726420506c6163652054726f706879 + 1 19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07.487964726120446f6f6d202d20326e6420506c6163652054726f706879 + 1 1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11.2331 + 1 6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a.2331 + 1 cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b.2331 + 69918000000 d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4.0014df105553444d + 1 dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee.487964726120446f6f6d202d2034746820506c6163652054726f706879 + TxOutDatumNone

Looks like the code I added reorders the UTxO so in the end you always get one utxo although the faucet had two UTxO which both could be used for publishing scripts.

I need to print the state of the UTxO before and after as well as txs if I am going to resolve this problem.
This blockfrost work seems neverending but we'll get there.

2025-04-29

SB on hydra-node using blockfrost

I am at point where I wrote a test in the DirectChainSpec to open, close and fanout a head using blockfrost.
~~The test is not entirely using blockfrost since we also rely on cardano-node on preview for querying UTxO for example.~~
The problem is that sometimes faucet UTxO comes back empty which is weird since I see at least one UTxO with large amount of ada but also some NFT's.
Why queryUTxO fails to return the correct UTxO? I feel like this problem is not part of the thing I am trying to solve but I definitely need a green test otherwise I can't know if the chain following logic for blockfrost works.
Maybe I should test this in e2e scenario instead and see how things behave in there? ~~Is it possible that cardano-node is not in sync completely before test is ran?~~
One thing I noticed - I am using blockfrost with faucet key to publish scripts at the beginning but I am not awaiting for the local cardano-node to see these transactions!
Then when I query the faucet UTxO I either see empty UTxO or get BadInputs error so I think I need to await for script publishing transactions definitely.
This is far from optimal - perhaps I need to create equivalent functions that work with blockfrost api instead?
After re-mapping all needed functions into blockfrost versions and not use local cardano-node for anything, I still get BadInputs error..hmm. At least I see the correct UTxO picked up so I'll work my way from there. This is probably some logic in the UTxO seed...
After adding blockfrost equivalent for awaitForTransaction I am still at the same place - makes sense. The produced output is not a problem but the actual transaction.
I pretty printed the faucet utxo and the tx and I don't see anything weird.
Important note is - we get a valid tx when building but the Blockfrost returns an error when submitting!
Decided to find just a singe utxo that is responsible from seeding from a faucet just to reduce a clutter but the error is the same:

Faucet UTxO: 815e52d1ee#0 ↦ 54829439 lovelace
62fb023528#1 ↦ 9261435223 lovelace
70e6c21881#1 ↦ 129433058019 lovelace + 1 13d1f7feab83ff4db444bf96b8677949c5bf9c709671f30ff8f33ab3.487964726120446f6f6d202d2033726420506c6163652054726f706879 + 1 19c98d04cdb6e1e782a73e693697d4a46ca9820d5d490a3bf6470a07.487964726120446f6f6d202d20326e6420506c6163652054726f706879 + 1 1a22028742629f3cf38b3d1036a088fea59eb30237a675420fb25c11.2331 + 1 6d92350897706b14832c62c5b5644e918f0b6b3b63ffc00a1a463828.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 ad39d849181dc206488fd726240c00b55547153ffdca8c079e1e34d9.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 bfe4ab531fd625ef33ea355fd85953eb944bffa401af767666ff411c.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 c953682b6eb5891c0bda35718c5261587d57e5e408079cbeb8cf881a.2331 + 1 cd6076d9d0098da4c7670c08f230e4efe31d666263c9db5196805d6e.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 d0c91707d75011026193c0fce742443dde66fa790936981ece5d9f8b.2331 + 69918000000 d8906ca5c7ba124a0407a32dab37b2c82b13b3dcd9111e42940dcea4.0014df105553444d + 1 dd7e36888a487f8b27687f65abd93e6825b4eb3ce592ee5f504862df.487964726120446f6f6d202d2031737420506c6163652054726f706879 + 1 fa10c5203512eeeb92bf79547b09f5cdb2e008689864b0175cca6fee.487964726120446f6f6d202d2034746820506c6163652054726f706879
Found UTxO: 62fb023528#1 ↦ 9261435223 lovelace
"f99907e0b4e3c9d554a68e76c3a72b4090cffb5c12d0cd471e29e1d0fa7184d2"

== INPUTS (1)
- cd62585298998cd809f6fe08a4af3087dab8f73ed67132b8c8fd4162fb023528#1
      ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "9783be7d3c54f11377966dfabc9284cd6c32fca1cd42ef0a4f1cc45b"})) StakeRefNull
      9261435223 lovelace
      TxOutDatumNone
      ReferenceScriptNone

== COLLATERAL INPUTS (0)

== REFERENCE INPUTS (0)

== OUTPUTS (2)
Total number of assets: 1
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d"})) StakeRefNull
      100000000 lovelace
      TxOutDatumNone
- ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "9783be7d3c54f11377966dfabc9284cd6c32fca1cd42ef0a4f1cc45b"})) StakeRefNull
      9161262858 lovelace
      TxOutDatumNone

== TOTAL COLLATERAL
TxTotalCollateralNone

== RETURN COLLATERAL
TxReturnCollateralNone

== FEE
TxFeeExplicit ShelleyBasedEraConway (Coin 172365)

== VALIDITY
TxValidityNoLowerBound
TxValidityUpperBound ShelleyBasedEraConway Nothing

== MINT/BURN
0 lovelace

== SCRIPTS (0)
Total size (bytes):  0

== DATUMS (0)

== REDEEMERS (0)

== REQUIRED SIGNERS
[]

== METADATA
TxMetadataNone

  can open, close & fanout a Head using Blockfrost [✘]

Failures:

  test/Test/DirectChainSpec.hs:385:3:
  1) Test.DirectChain can open, close & fanout a Head using Blockfrost
       uncaught exception: APIBlockfrostError
       BlockfrostError "BlockfrostBadRequest \"{\\\"contents\\\":{\\\"contents\\\":{\\\"contents\\\":{\\\"era\\\":\\\"ShelleyBasedEraConway\\\",\\\"error\\\":[\\\"ConwayUtxowFailure (UtxoFailure (ValueNotConservedUTxO (MaryValue (Coin 0) (MultiAsset (fromList []))) (MaryValue (Coin 9261435223) (MultiAsset (fromList [])))))\\\",\\\"ConwayUtxowFailure (UtxoFailure (BadInputsUTxO (fromList [TxIn (TxId {unTxId = SafeHash \\\\\\\"cd62585298998cd809f6fe08a4af3087dab8f73ed67132b8c8fd4162fb023528\\\\\\\"}) (TxIx {unTxIx = 1})])))\\\"],\\\"kind\\\":\\\"ShelleyTxValidationError\\\"},\\\"tag\\\":\\\"TxValidationErrorInCardanoMode\\\"},\\\"tag\\\":\\\"TxCmdTxSubmitValidationError\\\"},\\\"tag\\\":\\\"TxSubmitFail\\\"}\""

I don't see anything wrong with the tx but blockfrost seems to thing the input is invalid for whatever reason. I'll explore tx endpoints to see if I can find something useful.
Checked the mapping between blockfrost/cardano when creating UTxO and all looks good.
Added awaitForTransaction - blockfrost variant which didn't help out with the error.
Made all functions in Blockfrost.Client work in BlockfrostClientT IO so I can run them all from outside (I suspected that calling multiple blockfrost connections can cause problems) and this didn't help either.
I think all these changes I did are good to keep but I still don't see why submitting of a seeding tx fails?

2025-04-28

SN on fixing deposits using behavior tests

Fixing behavior spec for deposits in future, but to soon requires us to check the deadline.. but what is the upper bound? Re-use contestation period or introduce a deposit period (already)?
Decide to use the contestation period to limit the upper bound now (no need for a new option right now)
Switching to a realistic contestation period had some other behavior specs fail? The ones with 60 deadlines were easy to fix with a newDeadlineFarEnoughAway, but surprisingly the "close with decommit in flight" was failing?
Switching the bespoke test to assert DecommitFinalized revealed that this server output is not actually yielded when the head was closed right after?
When trying to fix assertions on "can close with decommit in flight" (by also removing the wait for contestation), I realized that the DecommitFinalized for the correctly approved decommit was never sent because the Head is already in Closed state when handling the OnDecrementTx observation - is thes test suite not simulating blocks correctly or can this really happen?
Indeed, the simulated chain of the BehaviorSpec is just delaying observation by block time. As the decrement is only posted after approval, but the test client directly submits Close after the Decommit command, it is possible that the OnDecrementTx comes after the OnCloseTx.. but this is impossible using a proper chain.
We should look into using MockChain also in BehaviorSpec (we did not have that when originally writing this suite)
The cardano-ledger bailing on generated utxo is really annoying… went down a rabbit hole to have a test of our generators at least.

2025-04-26

SN refocusing on deposit fixes

Pivot: Originally thought I want a model-based test as I need rollbacks (using MockChain). The property I would have liked to test would be something like: "In an open head, a deposit only results in utxo available on L2 if not rolled back." However, the rollbacks in this property would need to be bounded and related to the deposit deadline. i.e. if the deposit was 150 blocks out from when posted, only rollbacks over 150 blocks should not result in the utxo added. This is not only hard, but maybe not as useful as I thought.
Realization: we don't need to talk about rollbacks if we make "settlement" first class in any property we want to express. For example, we can say "A deposit is only processed after settled" which implies a notion of settlement. Practically this could be a --deposit-period which we can configure per node and we then just need to ensure the node behaves correctly off-chain. Hence, statements like this can be expressed and tested using less involved test suites like the BehaviorSpec and notably without rollbacks.
Besides behavior properties like these, we then only need "deposits are correctly observed" in which we need to ensure that our implementation reports deposit time (or slot?) and deadline honestly.
This is similar to "only proper head is observed", i.e. we would generate happy deposit txs and ensure through mutations that all necessary things are still in place for it to be observeDepositTx correctly. For example: a proper deposit tx would contain an upper validity to record "now". Not having that set should make the deposit not observed.
Could define those "observation tests" also as part of hydra-tx ContractSpec? Are those "healty" + mutated generated transactions comparable in quality to StateSpec - which we then could get rid of?

2025-04-24

SN on model based testing interlude

Want to reproduce a specific model test by copying the action ... output. Unfortunately the show instances are not aligning well with available data constructors :(
After fixing some show instances, I could derive a unit test from the model spec using the action sequence.
Now I see again a Deposit with directly expired deadline not resulting in a utxo added and a following NewTx failing. I need to distinguish in the nextState whether a Deposit is expected to be adding a utxo or not. Could also do this in a second action to be more flexible in when to check whether a deposit already succeeded?
The incorrectly bound variables in the counter examples was because of incomplete HasVariables instances!
Made the model check pass again by not adding the deposited utxo directly to the model and also not wait for CommitFinalized in performDeposit. What is a good Actions api to do the assertions we want?
When I want to check whether a txid created by Deposit really happened, then I need to use TxIdType Tx in a symbolic Var .. which feels a bit add as other things we simplified to using Payment? Is there a way to switch return value types between model and run model?
Realized seems to be exactly this?
Had issus with the arbitraryQ of UTCTime always being way in the past. Also, the perform of DepositIncremented is annoying as we can easily miss the CommitFinalized and looking into historic serverOutputs is messy..

2025-04-23

SN on fixing deposits

How to deal with incompatible deposits? We do observe them, but should the head logic track them?
When introducing a currentTime to the open state (in order to determine deadline being good or not) I realize that Tick contents would be useful to have on the Observation chain event, which would be easily possible. How is the contestation deadline done?
- Aha! The need for tracking a currentTime : UTCTime in the HeadState can be worked around by tracking all deposits and discard them on tick.
- Hm.. but that would move the decision whether to snapshot a pending deposit only to the Tick handling. Which means that it may only happen on the next block..
- But this is where the deposit tracking needs to go anyways .. we will never issue a snapshot directly when observing the deposit (that's why we are here) and if we decouple the issuance from the observation, the logic needs to go to the Tick handling anyways!
When changing CommitRecorded I stumble over newLocalUTxO = localUTxO <> deposited .. why would we want to update our local ledger already when recording the deposit!? This was likely a bug too..
Specification will be quite different than what we need to implement: there are no deposits tracked and only a wait for any previous pending deposits. To what level do we need to specify the logic of delaying deposits and checking deadlines?
Why was the increment snapshotting only waiting for an "unresolved Decommit" before requesting a snapshot?
Why do we need to wait at all (for other decommits or commits) if there is no snapshot in flight and we are the leader.. why not just snapshot what we want?
After moving the incremental commit snapshot decision to Tick handling, the model fails because of a NewTx can't spend a UTxO added through a Deposit before -> interesting!
After bringing back a Uα equivalent to the HeadLogic the model spec consistently finds an empty utxoToCommit which fails to submit an incrementTx -> good!
Interestingly, the model allows to do action $ Deposit {headIdVar = var2, utxoToDeposit = [], deadline = 1864-06-16 04:36:38.606749385646 UTC} which obviously results in an empty utxo to commit.. this can happen in the wild too!
- Unclear where exactly we want to deal with empty deposits.
Back to where we started with a very old Deposit and the node trying to do an increment with deadline already passed. This should be easy to fix by just not trying to snapshot it. However, what if a dishonest hydra-node would do just that? Would we approve that snapshot? Certainly the on-chain script would forbid it, but this could stall the head.
- This is similar to the empty utxo thing. While we can make our honest hydra-node do funky stuff, we must ensure that we do not sign snapshots that are funky!
- Which tests would best capture this? The ModelSpec won't see these issues once our honest implementation stops requesting funky snapshots!
To determine whether a deposit is still (or already) fine, we are back to needing a notion of UTCTime when doing that decision? We could do that updating in the Tick handling and keep information about a deposit being Outdated or so. Then, the snapshot acknowledgment code can tell whether a deposit is valid and only sign if it is.
- Tracking a full Deposit type in pendingDeposits which has a DepositStatus.
- With the new Deposit type I can easily mark deposits as Expired and need to fix several behavior tests to put realistic deadlines. However, the observability in tests is lacking and I definitely need a DepositExpired server output to fix all tests.

The fact that we need to create state changes to have the state updated, but also want to see them applied before determining the next active deposit is maybe a hint to a more monadic way of writing the logic functions?
After adding the DepositActivated and DepositExpired state changes I was debugging why the "can process transactions while commit pending" is still not passing: This is injecting a deposit observation, then asserts CommitRecorded and directly submits a NewTx this will have a snapshot be requested and confirmed. This first snapshot, however, is not yet including the utxoToCommit of the deposit becaues the Deposit was not yet Active (we have not seen time passing).. so the next snapshot will need to settle it. However, the other party (the new snaphshot leader) never saw the deposit because we used injectChainEvent!

2025-04-22

SN on fixing deposits

Deposit fixes: How to test this situation? I need a test suite that includes the off-chain logic, but also allows control over rollbacks and spending inputs.
- Model based tests are not including incremental commits :(
- TxTraceSpec contains deposit/increment, but does only exercise the L1 related code
- The behavior tests do cover deposit/increment behavior, but deposit observations are only injected! So rollbacks would not cover them.
Lets bite the bullet.. at least the model-based MockChain could be easily adapated to do deposits in simulateCommit?
Ran into the same issue as we had on CI when shrinking was failingon partial !. Guarding the shrinkAction to only include actions if their party is still in the seed seems to fix this.. but now shrinking does not terminate?
- Detour on improving shrinking and counterexamples of that checkModel problem .. shifting back to fixing deposits.
After adding Deposit actions, implementing a simulateDeposit and adjusting some generators/preconditions, I consistently run into test failures with deadline <- arbitrary. This is already interesting! The hydra-node seems to still try to increment deposits with very far in the past (year 1864) deadlines -> first bug found and reproducible!

2025-04-09

SB on blockfrost wallet queries

After using blockfrost query to get all eras and try to construct EraHistory I was surprised to discover that using nonEmptyFromList fails.
I know for sure that I am not constructing empty list here so this is confusing.
Fond the example in the atlas repo https://atlas-app.io/ but those were also failing which is even more surprising.
When looking at the blockfrost query results I noticed there are multiple NetworkEraSummary that start and end with slot 0 which is surprising:

eras: [ NetworkEraSummary
    { _networkEraStart = NetworkEraBound
        { _boundEpoch = Epoch 0
        , _boundSlot = Slot 0
        , _boundTime = 0s
        }
    , _networkEraEnd = NetworkEraBound
        { _boundEpoch = Epoch 0
        , _boundSlot = Slot 0
        , _boundTime = 0s
        }
    , _networkEraParameters = NetworkEraParameters
        { _parametersEpochLength = EpochLength 4320
        , _parametersSlotLength = 20s
        , _parametersSafeZone = 864
        }
    }
, NetworkEraSummary
    { _networkEraStart = NetworkEraBound
        { _boundEpoch = Epoch 0
        , _boundSlot = Slot 0
        , _boundTime = 0s
        }
    , _networkEraEnd = NetworkEraBound
        { _boundEpoch = Epoch 0
        , _boundSlot = Slot 0
        , _boundTime = 0s
        }
    , _networkEraParameters = NetworkEraParameters
        { _parametersEpochLength = EpochLength 86400
        , _parametersSlotLength = 1s
        , _parametersSafeZone = 25920
        }
    }

After removing them I can parse EraHistory with success but the question is how to filter out values from blockfrost? Which are valid eras?
I'll try filtering all eras that start and end with slot 0
This worked - I reported what I found to the blockfrost guys
Now it is time to move forward and test if the wallet queries actually work
I picked one DirectChainTest and decided to alter it so it runs on preview using withCardanoNodeOnKnownNetwork but I get

  test/Test/DirectChainSpec.hs:124:3:
  1) Test.DirectChain can init and abort a 2-parties head after one party has committed
       uncaught exception: QueryException
       QueryProtocolParamsEncodingFailureOnEra (AnyCardanoEra AlonzoEra) "Error in $: key \"poolVotingThresholds\" not found"

It seems like re-mapping the protocol params from blockfrost fails on poolVotingThresholds.
This happens immediately when cardano-node reports MsgSocketIsReady

cardano-node --version
cardano-node 10.1.4 - linux-x86_64 - ghc-8.10
git rev 1f63dbf2ab39e0b32bf6901dc203866d3e37de08

I can see that this field exists in the conway-genesis.json in the tmp folder of a test run

SB on finalizing recover/decrement observations should not be conditional

After PR review comments from FT I wanted to add one suggestion and that is to see the Head closed and finalized after initially committing and then decommitting some UTxO.
This leads to H28 error on close and this means we tried to close with initial snapshot but in fact we already got the confirmed snapshot.
When inspecting the logs I found out that the node, after a restart, does not observe any SnapshotConfirmed and therefore tries to close with initial one which fails.
Question is: Why did the restarted node failed to re-observe confirmed snapshot event?
Added some test code to wait and see SnapshotConfirmed in the restarted node to confirm it actually sees this event happening and the test fails exactly at this point.
When both nodes are running I can view the snapshot confirmed message is there but after a restart - node fails to see SnapshotConfirmed message again.
In the logs for both node 1 and 2 before restart I see two SnapshotConfirmed messages but in the restarted node these events are gone.
I realized the close works if I close from node that was not restarted but what I want to do is wait for the restarted node to catch up and then close.
I removed fiddling with the recover and wanted to get this basic test working but closing with restarted node, even after re-observing the last decommit, fails with H28 FailedCloseInitial.
This means the restarted node tried to close with the initial snapshot but one of the values doesn't match. We expect the version to be 0, snapshot number to be 0 and utxo hash should match the initial one.
last-known-revision for both nodes before I shutdown one of them is 11 but the restarted node, after removing the last-known-revision file ends up having value 13. How come it received more messages?
When comparing the state files I see discrepancies in eventId and the restarted node has a DecommitRecorded as the last event (other than ticks)
Regular node decommit recorded:

{"eventId":44,"stateChanged":{"decommitTx":{"cborHex":"84a300d9010281825820ad7458781dc19e427fca77c8c7b2db1b56c81c11590e2ae3999f2f13db8c51c200018182581d60f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d1a004c4b400200a100d9010281825820eb94e8236e2099357fa499bfbc415968691573f25ec77435b7949f5fdfaa5da0584071b6c5956083ff7ac7ad49d5a75c77967b5ad2e7fd756c1de226f71cdf89e5d383bc88975c9ca7deab135f4ea9014666aa0e257f26bdd94dda2df60c922e9306f5f6","description":"","txId":"3095040e42ed9b193f8a66699b1631c17a85f670aee3c4d77fb3cfb195ea6bcb","type":"Tx ConwayEra"},"headId":"654b2b0e5ff3e0a902a12918b63628cdd478364caa4f0c758e6f7490","newLocalUTxO":{},"tag":"DecommitRecorded","utxoToDecommit":{"3095040e42ed9b193f8a66699b1631c17a85f670aee3c4d77fb3cfb195ea6bcb#0":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":5000000}}}},"time":"2025-04-10T07:30:58.882632162Z"}

Restarted node decommit recorded

{"eventId":76,"stateChanged":{"decommitTx":{"cborHex":"84a300d9010281825820ad7458781dc19e427fca77c8c7b2db1b56c81c11590e2ae3999f2f13db8c51c200018182581d60f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d1a004c4b400200a100d9010281825820eb94e8236e2099357fa499bfbc415968691573f25ec77435b7949f5fdfaa5da0584071b6c5956083ff7ac7ad49d5a75c77967b5ad2e7fd756c1de226f71cdf89e5d383bc88975c9ca7deab135f4ea9014666aa0e257f26bdd94dda2df60c922e9306f5f6","description":"","txId":"3095040e42ed9b193f8a66699b1631c17a85f670aee3c4d77fb3cfb195ea6bcb","type":"Tx ConwayEra"},"headId":"654b2b0e5ff3e0a902a12918b63628cdd478364caa4f0c758e6f7490","newLocalUTxO":{},"tag":"DecommitRecorded","utxoToDecommit":{"3095040e42ed9b193f8a66699b1631c17a85f670aee3c4d77fb3cfb195ea6bcb#0":{"address":"addr_test1vru2drx33ev6dt8gfq245r5k0tmy7ngqe79va69de9dxkrg09c7d3","datum":null,"datumhash":null,"inlineDatum":null,"inlineDatumRaw":null,"referenceScript":null,"value":{"lovelace":5000000}}}},"time":"2025-04-10T07:31:02.301798566Z"}

Let's try to see the decommit timeline between two states (I am aware these event's do not need to be in order but I think etcd should deliver in order after restart)
So let's track this decommit between two nodes


DecommitRecorded 
running node        2025-04-10T07:30:58.882632162Z
restarted node      2025-04-10T07:31:02.301798566Z

DecommitApproved 
running node        2025-04-10T07:30:58.894604418Z
restarted node      missing event

DecommitFinalized  
running node        2025-04-10T07:30:59.007515339Z
restarted node      2025-04-10T07:31:02.300503374Z

So it seems like the restarted node is late couple of seconds but how can it be that in the test we wait to see DecommitFinalized and if we try to close after the restarted node still thinks it is at version 0?

2025-04-04

SN on exploring `dingo`

Trying out dingo and whether I could hook it up to hydra-node
When synchronizing preview with dingo the memory footprint was growing as sync progressed, but did not increase to same level when restarting the chain sync (althoug it picked up the starting slot etc.)
The system was swapping a lot of memory too (probably reached max of my 32GB)
Querying address of latest hydra head address shows two heads on preview, but our explorer only shows one?

Querying the dingo node seems to work, but I get a hydra scripts discovery error?

MissingScript {scriptName = "\957Initial", scriptHash = "c8a101a5c8ac4816b0dceb59ce31fc2258e387de828f02961d2f2045", discoveredScripts = fromList ["0e35115a2c7c13c68ecd8d74e4987c04d4539e337643be20bb3274bd"]}

Indeed dingo behaves slightly different on the queryUTxOByTxIn local state query: when requesting three txins, it only responds with one utxo

[ TxIn "b7b88533de303beefae2d8bb93fe1a1cd5e4fa3c4439c8198c83addfe79ecbdc" ( TxIx 0 ) , TxIn "da1cc0eef366031e96323b6620f57bc166cf743c74ce76b6c3a02c8f634a7d20" ( TxIx 0 ) , TxIn "6665f1dfdf9b9eb72a0dd6bb73e9e15567e188132b011e7cf6914c39907ac484" ( TxIx 0 ) ] returned utxo: 1

After fixing that to query three times, the next stop gap seems to come from chain sync:
```
bearer closed: "<socket: 23> closed when reading data, waiting on next header True"
```

Maybe something on the n2c handshake does not work? On dingo side I see:

{"time":"2025-04-05T13:47:05.495636842+02:00","level":"INFO","msg":"listener: accepted connection from unix@629","component":"connmanager"} {"time":"2025-04-05T13:47:05.4957064+02:00","level":"ERROR","msg":"listener: failed to setup connection: could not register protocol with muxer","component":"connmanager"}

When debugging how far we get on the handshake protocol I learn how gouroboros implements the state transitions of the miniprotocols using StateMap.
I realize that now the query for scripts not even works.. maybe my instrumentation broke something? Also.. all my instrumentation happened on vendored code in vendor/ of the dingo repo. I wonder how developers do the editing most convenient in this setup?
The Chain.Direct switch to connectToLocalNodeWithVersions was problematic, now it fetches the scripts correctly and the chain sync starts
It's definitely flaky in how "far" we get.. maybe the dingo node is only accepting n2c connections while connected upstream on n2n (I have been in a train with flaky connection).

Once it progressed now onto a RollForward where the queryTimeHandle would query the EraHistory and fail time conversion with error:

TimeConversionException {slotNo = SlotNo 77202345, reason = "PastHorizon {pastHorizonCallStack = [(\"runQuery\",SrcLoc {srcLocPackage = \"ouroboros-consensus-0.22.0.0-f90d7bc7c4431d706016c293a932800b9c1e28c3b268597acc5b945a9be83125\", srcLocModule = \"Ouroboros.Consensus.HardFork.History.Qry\", srcLocFile = \"src/ouroboros-consensus/Ouroboros/Consensus/HardFork/History/Qry.hs\", srcLocStartLine = 439, srcLocStartCol = 44, srcLocEndLine = 439, srcLocEndCol = 52}),(\"interpretQuery\",SrcLoc {srcLocPackage = \"hydra-node-0.21.0-inplace\", srcLocModule = \"Hydra.Chain.Direct.TimeHandle\", srcLocFile = \"src/Hydra/Chain/Direct/TimeHandle.hs\", srcLocStartLine = 91, srcLocStartCol = 10, srcLocEndLine = 91, srcLocEndCol = 24}),(\"slotToUTCTime\",SrcLoc {srcLocPackage = \"hydra-node-0.21.0-inplace\", srcLocModule = \"Hydra.Chain.Direct.TimeHandle\", srcLocFile = \"src/Hydra/Chain/Direct/TimeHandle.hs\", srcLocStartLine = 86, srcLocStartCol = 7, srcLocEndLine = 86, srcLocEndCol = 20}),(\"mkTimeHandle\",SrcLoc {srcLocPackage = \"hydra-node-0.21.0-inplace\", srcLocModule = \"Hydra.Chain.Direct.TimeHandle\", srcLocFile = \"src/Hydra/Chain/Direct/TimeHandle.hs\", srcLocStartLine = 116, srcLocStartCol = 10, srcLocEndLine = 116, srcLocEndCol = 22})], pastHorizonExpression = Some (EPair (ERelToAbsTime (ERelSlotToTime (EAbsToRelSlot (ELit (SlotNo 77202345))))) (ESlotLength (ELit (SlotNo 77202345)))), pastHorizonSummary = [EraSummary {eraStart = Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 0, boundEpoch = EpochNo 0}, eraEnd = EraEnd (Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 0, boundEpoch = EpochNo 0}), eraParams = EraParams {eraEpochSize = EpochSize 4320, eraSlotLength = SlotLength 20s, eraSafeZone = StandardSafeZone 0, eraGenesisWin = GenesisWindow {unGenesisWindow = 0}}},EraSummary {eraStart = Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 0, boundEpoch = EpochNo 0}, eraEnd = EraEnd (Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 0, boundEpoch = EpochNo 0}), eraParams = EraParams {eraEpochSize = EpochSize 86400, eraSlotLength = SlotLength 1s, eraSafeZone = StandardSafeZone 0, eraGenesisWin = GenesisWindow {unGenesisWindow = 0}}},EraSummary {eraStart = Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 0, boundEpoch = EpochNo 0}, eraEnd = EraEnd (Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 0, boundEpoch = EpochNo 0}), eraParams = EraParams {eraEpochSize = EpochSize 86400, eraSlotLength = SlotLength 1s, eraSafeZone = StandardSafeZone 0, eraGenesisWin = GenesisWindow {unGenesisWindow = 0}}},EraSummary {eraStart = Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 0, boundEpoch = EpochNo 0}, eraEnd = EraEnd (Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 0, boundEpoch = EpochNo 0}), eraParams = EraParams {eraEpochSize = EpochSize 86400, eraSlotLength = SlotLength 1s, eraSafeZone = StandardSafeZone 0, eraGenesisWin = GenesisWindow {unGenesisWindow = 0}}},EraSummary {eraStart = Bound {boundTime = RelativeTime 0s, boundSlot = SlotNo 172800, boundEpoch = EpochNo 2}, eraEnd = EraEnd (Bound {boundTime = RelativeTime 259200s, boundSlot = SlotNo 86400, boundEpoch = EpochNo 1}), eraParams = EraParams {eraEpochSize = EpochSize 86400, eraSlotLength = SlotLength 1s, eraSafeZone = StandardSafeZone 0, eraGenesisWin = GenesisWindow {unGenesisWindow = 0}}},EraSummary {eraStart = Bound {boundTime = RelativeTime 259200s, boundSlot = SlotNo 55728000, boundEpoch = EpochNo 645}, eraEnd = EraEnd (Bound {boundTime = RelativeTime 55814400s, boundSlot = SlotNo 345600, boundEpoch = EpochNo 4}), eraParams = EraParams {eraEpochSize = EpochSize 86400, eraSlotLength = SlotLength 1s, eraSafeZone = StandardSafeZone 0, eraGenesisWin = GenesisWindow {unGenesisWindow = 0}}},EraSummary {eraStart = Bound {boundTime = RelativeTime 55814400s, boundSlot = SlotNo 77155200, boundEpoch = EpochNo 893}, eraEnd = EraEnd (Bound {boundTime = RelativeTime 77241600s, boundSlot = SlotNo 55900800, boundEpoch = EpochNo 647}), eraParams = EraParams {eraEpochSize = EpochSize 86400, eraSlotLength = SlotLength 1s, eraSafeZone = StandardSafeZone 0, eraGenesisWin = GenesisWindow {unGenesisWindow = 0}}}]}"}

I saw that same error when using cardano-cli query tip .. seems like the era history local state query is not accurately reporting epoch bounds.
I conclude that dingo is easy to use and navigate around, but the N2C API is not complete yet. Maybe my work on the LocalStateQuery API in cardano-blueprint could benefit the project and making gouroboros more conformant (at least from a message serialization point of view).

March 2025

2025-03-28

SN on memory leak

Non-profiled Haskell binaries can be inspected using -s and -hT RTS arguments

Running the hydra-node using a 2GB state file as provided by GD the node will load the state and then fail on mismatched keys (as we have not the right ones):

 151,712,666,608 bytes allocated in the heap
  14,411,335,656 bytes copied during GC
     973,747,296 bytes maximum residency (53 sample(s))
      24,460,192 bytes maximum slop
            2033 MiB total memory in use (0 MiB lost due to fragmentation)

The peekForeverE in https://github.com/cardano-scaling/hydra/pull/1919 seem not to make any difference:

 151,712,692,632 bytes allocated in the heap
  14,409,258,352 bytes copied during GC
     973,732,032 bytes maximum residency (53 sample(s))
      24,545,088 bytes maximum slop
            2033 MiB total memory in use (0 MiB lost due to fragmentation)

Using hT a linear growth of memory can be seen quite easily.
First idea: lastEventId conduit was using foldMapC which might be building thunks via mappend
- Nope, that was not the issue.
That was not the issue.. disabling aggregation of chainStateHistory and only load headState next.
- Still linear growth.. so the culprit most likely is inside the main loading of headState (besides other issues?)
Let's turn on StrictData on all of HeadLogic as a first stab at seeing more stricture usage of HeadState et al.
This works! Making HeadLogic{.State, .Outcome} all StrictData already pushes the heap usage down ~5MB!
Possible explanation: With gigabytes of state updates we have almost exclusively TransactionReceived et al state changes. In the aggregate we usually build up thunks like allTxs = allTxs <> fromList [(txId tx, tx)] which will leak memory until forced into one concrete list when showing the HeadState first (which will probably collapse the memory usage again).

With StrictData we have a maximum residency of 10MB after loading 2GB of state events:

 152,176,815,256 bytes allocated in the heap
  16,702,572,088 bytes copied during GC
       9,967,848 bytes maximum residency (2387 sample(s))
         215,600 bytes maximum slop
              43 MiB total memory in use (0 MiB lost due to fragmentation)

Trying to narrow in exact source of memory leak so I do not need to put bangs everywhere
- allTxs and localTxs assignments are not the source of it .. maybe the coordinatedHeadState record update?
- No .. also not really. Maybe it's time to recompile with profiling enabled and make some coffee (this will take a while).
- When using profiling: True using the haskell.nix managed dependencies, I ran into this error:
- Setting enableProfiling = true in the haskell.nix project modules rebuilds the whole world, but that is expected.
- Hard to spot where exactly we are creating the space leak / thunks. This blog post is helpful still: http://blog.ezyang.com/2011/06/pinpointing-space-leaks-in-big-programs/
- I am a bit confused why so many of the cost center point to parsing and decoding code .. maybe the transactions themselves (which make up the majority of data) are not forced for long? This would make sense because the HeadLogic does not inspect transactions themselves (much).
- Only strictness annotations on a !tx did not help, but let's try a StrictData on StateChanged
- StrictData on HeadLogic.Outcome does not fix it … so it must be something related to the HeadState.
- The retainer profile actually points quite clearly to aggregate.
- The biggest things on the heap are bytes, thunks and types related to a cardano transaction body.
Going back to zero in on branches of aggregate via exclusion
- Disabling all CoordinatedHeadState modifications makes memory usage minimal again
- Enabling SnapshotConfirmed -> still bounded
- Enabling PartySignedSnapshot -> still bounded
- Enabling SnapshotRequested -> growing!
- Without allTxs update -> bounded!
- This line creates thunks!? allTxs = foldr Map.delete allTxs requestedTxIds
- Neither, forcing allTxs nor requestedTxIds helped
- Is it really only this line? enableing all other aggregate updates to CoordinatedHeadState
- It's both allTxs usages
- If we only make allTxs field strict? -> Bounded!

2025-03-27

SB on fanout utxo bug

After easy changes to FanoutTx to include observed UTxO instead of using the confirmed snapshot there are problems in the DirectChainSpec and Model.
Let's look at DirectChainSpec first - I need to come up with a utxo value for this line here:

aliceChain `observesInTime` OnFanoutTx headId mempty

Failed test looks like this:

  test/Test/DirectChainSpec.hs:578:35:
  1) Test.DirectChain can open, close & fanout a Head
       expected: OnFanoutTx {headId = UnsafeHeadId "eK+\SO_\243\224\169\STX\161)\CAN\182\&6(\205\212x6L\170O\fu\142ot\144", fanoutUTxO = fromList [(TxIn "0762c8de902abe1e292e691066328c932d95e29c9a564d466e8bc791527e359f" (TxIx 0),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "8163bc1d679f90d073784efdc761288dbc2dc21a352f69238070fc45"})) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 2000000) (MultiAsset (fromList [])))) TxOutDatumNone ReferenceScriptNone),(TxIn "c9a733c945fdb7819648a58d7d6b9a30af2ac458a27f5bb7e9c41f92da82ba2c" (TxIx 0),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "8163bc1d679f90d073784efdc761288dbc2dc21a352f69238070fc45"})) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 2000000) (MultiAsset (fromList [])))) TxOutDatumNone ReferenceScriptNone)]}
        but got: OnFanoutTx {headId = UnsafeHeadId "eK+\SO_\243\224\169\STX\161)\CAN\182\&6(\205\212x6L\170O\fu\142ot\144", fanoutUTxO = fromList [(TxIn "880c3d807a48d432788158f879a81a5ddc6c1ad6527fe70922175e621ea08092" (TxIx 0),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (ScriptHashObj (ScriptHash "0e35115a2c7c13c68ecd8d74e4987c04d4539e337643be20bb3274bd")) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 4879080) (MultiAsset (fromList [(PolicyID {policyID = ScriptHash "654b2b0e5ff3e0a902a12918b63628cdd478364caa4f0c758e6f7490"},fromList [("4879647261486561645631",1),("f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d",1)])])))) (TxOutDatumInline BabbageEraOnwardsConway (HashableScriptData "\216{\159\216y\159X\FSeK+\SO_\243\224\169\STX\161)\CAN\182\&6(\205\212x6L\170O\fu\142ot\144\159X \213\191J?\204\231\ETB\176\&8\139\204'I\235\193H\173\153i\178?E\238\ESC`_\213\135xWj\196\255\216y\159\EM'\DLE\255\NUL\SOHX \193\211\DC4E\234\252\152\157\239\186\RSmVF\141\208\218\135\141\160{\fYFq\245\SOH\148\nOS\DC1X \227\176\196B\152\252\FS\DC4\154\251\244\200\153o\185$'\174A\228d\155\147L\164\149\153\ESCxR\184UX \227\176\196B\152\252\FS\DC4\154\251\244\200\153o\185$'\174A\228d\155\147L\164\149\153\ESCxR\184U\128\ESC\NUL\NUL\SOH\149\214\218\152\136\255\255" (ScriptDataConstructor 2 [ScriptDataConstructor 0 [ScriptDataBytes "eK+\SO_\243\224\169\STX\161)\CAN\182\&6(\205\212x6L\170O\fu\142ot\144",ScriptDataList [ScriptDataBytes "\213\191J?\204\231\ETB\176\&8\139\204'I\235\193H\173\153i\178?E\238\ESC`_\213\135xWj\196"],ScriptDataConstructor 0 [ScriptDataNumber 10000],ScriptDataNumber 0,ScriptDataNumber 1,ScriptDataBytes "\193\211\DC4E\234\252\152\157\239\186\RSmVF\141\208\218\135\141\160{\fYFq\245\SOH\148\nOS\DC1",ScriptDataBytes "\227\176\196B\152\252\FS\DC4\154\251\244\200\153o\185$'\174A\228d\155\147L\164\149\153\ESCxR\184U",ScriptDataBytes "\227\176\196B\152\252\FS\DC4\154\251\244\200\153o\185$'\174A\228d\155\147L\164\149\153\ESCxR\184U",ScriptDataList [],ScriptDataNumber 1743066405000]]))) ReferenceScriptNone)]}

So it seems like there is a script output in the observed UTxO with 4879080 lovelace, some tokens and seems like this is a head output and what we expect is distributed outputs to hydra-node parties containing the fanout amount.
These head assets that I see should have been burned already? We get this utxo in the observation using let inputUTxO = resolveInputsUTxO utxo tx
If I use

  (headInput, headOutput) <- findTxOutByScript inputUTxO Head.validatorScript
  UTxO.singleton (headInput, headOutput)

then the utxo is the same which is expected.

How come the fanout tx does not contain pub key outputs?
If I use utxoFromTx fanoutTx then I get the expected pub key outputs:

  1) Test.DirectChain can open, close & fanout a Head
       expected: OnFanoutTx {headId = UnsafeHeadId "eK+\SO_\243\224\169\STX\161)\CAN\182\&6(\205\212x6L\170O\fu\142ot\144", fanoutUTxO = fromList []}
        but got: OnFanoutTx {headId = UnsafeHeadId "eK+\SO_\243\224\169\STX\161)\CAN\182\&6(\205\212x6L\170O\fu\142ot\144", fanoutUTxO = fromList [(TxIn "431e45c0048e0aa104deaca1e8aca454c85efd71c52948e418d9119fd8cdf7b3" (TxIx 0),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "4e932840c5d2d3664237149fd3e9ba09c531581126fbdbab073c31ce"})) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 2000000) (MultiAsset (fromList [])))) TxOutDatumNone ReferenceScriptNone),(TxIn "431e45c0048e0aa104deaca1e8aca454c85efd71c52948e418d9119fd8cdf7b3" (TxIx 1),TxOut (AddressInEra (ShelleyAddressInEra ShelleyBasedEraConway) (ShelleyAddress Testnet (KeyHashObj (KeyHash {unKeyHash = "f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d"})) StakeRefNull)) (TxOutValueShelleyBased ShelleyBasedEraConway (MaryValue (Coin 90165992) (MultiAsset (fromList [])))) TxOutDatumNone ReferenceScriptNone)]}

but the overall test is red since we construct artificial TxIns in utxoFromTx

I created findPubKeyOutputs to match on all pub key outputs and then I see expected outputs but they also contain change output that returns some ada to the hydra-node wallet. Life is not simple.
In the end I changed all tests that match exactly on final utxo to make sure that subset of final utxo is there (disregarding the change output).
Changes in fanout observation boiled down to findPubKeyOutputs $ utxoFromTx tx

SB on memory leak on loading events from disk

Midnight people have reported that they still see some memory issues when loading a huge state file from disk.
The main problem is making sure the fix works, I still don't have a good idea on how to make sure my changes reduce the memory consumption.
Problem lies in this piece of code:

 (lastEventId, (headState, chainStateHistory)) <-
    runConduitRes $
      sourceEvents eventSource
        .| getZipSink
          ( (,)
              <$> ZipSink (foldMapC (Last . pure . getEventId))
              <*> ZipSink recoverHeadStateC
          )
...

recoverHeadStateC =
    mapC stateChanged
      .| getZipSink
        ( (,)
            <$> ZipSink (foldlC aggregate initialState)
            <*> ZipSink (foldlC aggregateChainStateHistory $ initHistory initialChainState)
        )

and of course the way we create PersistenceIncremental which is responsible for reading the file (sourceEvents eventSource part).

 sourceFileBS fp
          .| linesUnboundedAsciiC
          .| mapMC
            ( \bs ->
                case Aeson.eitherDecodeStrict' bs of
                  Left e -> ...
                  Right decoded -> ...
            )

Initially I noticed the usage of foldlC which is strict and thought perhaps this is the problem but could not find lazy alternative and in general I don't believe this is the real issue.
I am more keen to investigate this code:

 sourceFileBS fp
          .| linesUnboundedAsciiC
          .| mapMC ...

linesUnboundedAsciiC could be the cause since I believe it is converting the whole stream

Convert a stream of arbitrarily-chunked textual data into a stream of data
where each chunk represents a single line. Note that, if you have
unknownuntrusted input, this function is unsafe/, since it would allow an
attacker to form lines of massive length and exhaust memory.

I also found an interesting function peekForeverE that should Run a consuming conduit repeatedly, only stopping when there is no more data available from upstream.
Could I use benchmarks to simulate heavy load from disk?
I just tried running the benchmarks with and without one line change and it seems like the memory consumption is reduced
- BEFORE ->

Average confirmation time (ms): 57.599974154
P99: 76.48237684999998ms
P95: 67.55752405ms
P50: 56.9354805ms
Invalid txs: 0

### Memory data

 | Time | Used | Free |
|------|------|------|
 | 2025-03-27 15:39:59.474482067 UTC | 14.2G | 35.1G |
 | 2025-03-27 15:40:04.474412824 UTC | 14.4G | 34.9G |
 | 2025-03-27 15:40:09.474406479 UTC | 14.4G | 34.9G |
 | 2025-03-27 15:40:14.474403701 UTC | 14.4G | 34.8G |
 | 2025-03-27 15:40:19.47445777 UTC | 14.4G | 34.8G |
 | 2025-03-27 15:40:24.474392458 UTC | 14.4G | 34.8G |
 | 2025-03-27 15:40:29.474439923 UTC | 14.4G | 34.8G |
 | 2025-03-27 15:40:34.474408859 UTC | 14.5G | 34.7G |
 | 2025-03-27 15:40:39.474436556 UTC | 14.4G | 34.7G |
 | 2025-03-27 15:40:44.474414945 UTC | 14.5G | 34.7G |

Confirmed txs/Total expected txs: 300/300 (100.00 %)
Average confirmation time (ms): 9.364033643
P99: 19.919154109999997ms
P95: 15.478096ms
P50: 7.7630015ms
Invalid txs: 0

### Memory data

 | Time | Used | Free |
|------|------|------|
 | 2025-03-27 15:40:55.995225272 UTC | 14.2G | 35.1G |
 | 2025-03-27 15:41:00.995294779 UTC | 14.2G | 35.1G |
 | 2025-03-27 15:41:05.995309124 UTC | 14.2G | 35.1G |
 | 2025-03-27 15:41:10.995299687 UTC | 14.3G | 35.0G |
 | 2025-03-27 15:41:15.995284362 UTC | 14.3G | 35.0G |
 | 2025-03-27 15:41:20.995281122 UTC | 14.3G | 35.0G |

AFTER ->

Average confirmation time (ms): 57.095020378
P99: 72.8903286ms
P95: 66.89188805ms
P50: 57.172249ms
Invalid txs: 0

### Memory data

 | Time | Used | Free |
|------|------|------|
 | 2025-03-27 15:37:47.726878831 UTC | 13.7G | 35.6G |
 | 2025-03-27 15:37:52.726824668 UTC | 13.9G | 35.5G |
 | 2025-03-27 15:37:57.726768654 UTC | 14.0G | 35.3G |
 | 2025-03-27 15:38:02.72675874 UTC | 14.0G | 35.3G |
 | 2025-03-27 15:38:07.726756126 UTC | 14.0G | 35.3G |
 | 2025-03-27 15:38:12.726795633 UTC | 14.0G | 35.2G |
 | 2025-03-27 15:38:17.726793141 UTC | 14.1G | 35.2G |
 | 2025-03-27 15:38:22.726757309 UTC | 14.1G | 35.1G |
 | 2025-03-27 15:38:27.726764279 UTC | 14.1G | 35.1G |
 | 2025-03-27 15:38:32.726781991 UTC | 14.1G | 35.1G |

Confirmed txs/Total expected txs: 300/300 (100.00 %)
Average confirmation time (ms): 9.418157436
P99: 19.8506584ms
P95: 15.841609050000002ms
P50: 7.821248000000001ms
Invalid txs: 0

### Memory data

 | Time | Used | Free |
|------|------|------|
 | 2025-03-27 15:38:45.195815881 UTC | 13.8G | 35.6G |
 | 2025-03-27 15:38:50.195894922 UTC | 14.0G | 35.4G |
 | 2025-03-27 15:38:55.19592388 UTC | 13.8G | 35.5G |
 | 2025-03-27 15:39:00.195971592 UTC | 14.1G | 35.2G |
 | 2025-03-27 15:39:05.195891924 UTC | 14.3G | 35.0G |
 | 2025-03-27 15:39:10.195897911 UTC | 14.3G | 35.0G |

I think I could try to keep the state file after running the benchmarks and then try to start a hydra-node using this (hopefully huge) state file and then peek into prometheus metrics to observe reduced memory usage.
What I find weird is that the same persistence functions are used in the api server but there is no report leakage there - perhaps it boils down on how we consume this stream?
Managed to get a state file with over 300k events so let's see if we can measure reduced usage.
This is my invocation of hydra-node so I can copy paste it when needed:

./result/bin/hydra-node \
  --node-id 1 --api-host 127.0.0.1  \
  --monitoring-port 6000 \
  --hydra-signing-key /home/v0d1ch/code/hydra/memory/state-0/me.sk \
  --hydra-scripts-tx-id "8f46dbf87bd7eb849c62241335fb83b27e9b618ea4d341ffc1b2ad291c2ad416,25f236fa65036617306a0aaf0572ddc1568cee0bc14aee14238b1196243ecddd,59a236ac22eb1aa273c4bcd7849d43baddd8fcbc5c5052f2eb074cdccbe39ff4" \
  --cardano-signing-key /home/v0d1ch/code/hydra/memory/1.sk \
  --ledger-protocol-parameters /home/v0d1ch/code/hydra/memory/state-0/protocol-parameters.json \
  --testnet-magic 42 \
  --contestation-period 10 \
  --deposit-deadline 10 \
  --node-socket /home/v0d1ch/code/hydra/memory/node.socket \
  --persistence-dir /home/v0d1ch/code/hydra/memory/state-0

Didn't find the time to properly connect to some tool to measure the memory but by looking at the timestamps between LoadindState and LoadedState traces I can see that new changes give MUCH better performance:

With the current master:

start loading timestamp":"2025-03-27T17:12:08.57862623Z
loaded        timestamp":"2025-03-27T17:12:28.991870713Z

With one-liner change:   

start loading timestamp":"2025-03-27T16:58:54.055623085Z
loaded        timestamp":"2025-03-27T16:59:15.05648201Z

It looks like it took us 20 seconds to load around 335 mb state file and new change reduces this to around a second!

2025-03-24

SB on pending deposits bug

It seems like we have a bug in displaying our pending deposits where deposits that are already incremented or recovered still show when doing a request to hydra-node api /commits.
I extended one e2e test we had related to pending deposits and added one check after all others where I spin up again two hydra-nodes and call the endpoint to see if all pending deposits are cleared.

✦ ➜ cabal test hydra-cluster --test-options='--match="can see pending deposits" --seed 278123554'

The test seems flaky but in general it almost always fails.
From just looking at the code I couldn't see anything weird
Found one weird thing: I asserted that in the node-1 state file there are three CommitRecorded and three CommitRecovered but in node-2 state file there are two CommitRecovered missing.
Is the whole bug related to who does the recovering/recording?
The test outcome although red shows correct txids for the non-recovered txs
We only assert node-1 sees all CommitRecover messages but don't do it for the node-2 since that node is shut down at this point (in order to be able to prevent deposits from kicking in).
Is this a non-issue after all? I think so since we stop one node and then try to assert that after restart it sees some other commits being recovered but those were never recorded in the node local state. What is weird is that the test was flaky but using constant seed yields always the same results.
If a node fails to see OnIncrementTx then the deposit is stuck in pending local state forever.

2025-03-12

SB on Model tests weird behavior

Currently I had to sprinkle treadDelay here and there in the model tests since otherwise they hang for a long time and eventually (I think) report the shrinked values that fail the test.
This problem is visible mainly in CI where the resources available are not so big, locally the same tests pass.
If I remove the threadDelay the memory grows really big and I need to kill the process.
This started happening when I had to replace GetUTxO which no longer exists with queryState
I looked at this with NS and found out that we were missing to wait for all nodes to see a DecommitFinalized - we were waiting only on our node to see it. This seemed to have fixed the model test and was a bit surprising to be this easy since I expected a lot of problems in finding out what went wrong.

SB on figuring out what happened in our Head

The situation is that we are unable to close because of H13 MustNotChangeVersion
This happens because the version in the input datum (open datum) does not match with the version in the output (close datum).
Local state says I am on version 3 and onchain it seems the situation is the same - 3! But this can't be since the onchain check would pass then. This is how the datum looks https://preview.cexplorer.io/datum/8e4bd7ac38838098fbf23e5702653df2624bcfa4cf0c5236498deeede1fdca78
Looking at the state it seems like we try to close, the snapshot version contains correct version (3) but openVersion is still at 2:

  ...
                  "utxoToCommit": null,
                  "utxoToDecommit": null,
                  "version": 3
                },
                "tag": "ConfirmedSnapshot"
              },
              "headId": "50bb0874ae28515a2cff9c074916ffe05500a3b4eddea4178d1bed0b",
              "headParameters": {
                "contestationPeriod": 300,
                "parties": [
...
              "openVersion": 2,
              "tag": "CloseTx"
            },
            "tag": "OnChainEffect"
          }

Question is how did we get to this place? It must be that my node didn't observe and emit one CommmitFinalized which is when we do the version update - upon increment observation.
There are 24 lines with CommitFinalize message - only go up to version 2 while there are 36 lines with CommitRecorded - it seems like one recorded commit was not finalized for whatever reason.
OnIncrementTx shows up 8 times in the logs but in reality it is tied to only two increments so the third one was never observed.
OnDepositTx shows up 12 times in the logs but they are related to only two deposits.
Could it be that the decommit failed instead?
There is one DecommitRecorded and one DecommitFinalized so it seems good.
Seems like we have CommitRecorded for:
- "utxoToCommit":{"4b31dd7db92bde4359868911c1680ea28c0a38287a4e5b9f3c07086eca1ac26a#0"
- "utxoToCommit":{"4b31dd7db92bde4359868911c1680ea28c0a38287a4e5b9f3c07086eca1ac26a#1"
- "utxoToCommit":{"22cb19c790cd09391adf2a68541eb00638b8011593b3867206d2a12a97f4bf0d#0"
We received CommitFinalized for:
"theDeposit":"44fa1bc9b04d2ffee50fd84088517c3f7b530353834e7c678fdd05073881cb40"
- "theDeposit":"5b93f95068148482a1e27979517e8ab467f85e72551cfc9baaa2086a60e7353a"
So one commit was never finalized but it is a bit hard to connect recorded and finalized commits.
OnDepositTx was seen for txids: - 44fa1bc9b04d2ffee50fd84088517c3f7b530353834e7c678fdd05073881cb40 - 5b93f95068148482a1e27979517e8ab467f85e72551cfc9baaa2086a60e7353a - 83e7c36a9d4727e00169409f869d0f94737672c7e87850632b9efe1637f8ef8f
OnIncrementTx was seen for:
- 44fa1bc9b04d2ffee50fd84088517c3f7b530353834e7c678fdd05073881cb40
- 5b93f95068148482a1e27979517e8ab467f85e72551cfc9baaa2086a60e7353a so we missed to observe deposit `83e7c36a9d4727e00169409f869d0f94737672c7e87850632b9efe1637f8ef8f https://preview.cexplorer.io/tx/83e7c36a9d4727e00169409f869d0f94737672c7e87850632b9efe1637f8ef8f#data
Question is what to do with this Head? Can it be closed somehow?
We should query the deposit address to see what kind of UTxOs are available there.

2025-03-10

FT on SideLoad-Snapshot

Added an endpoint to GET the latest confirmed snapshot, which is needed to construct the side-load request, but it does not include information about the latest seen snapshot. Waiting on pull#1860 to enhance it.
In our scenario, the head got stuck on InitialSnapshot. This means that during side-loading, we must act similarly to clear pending transactions (pull#1840).
Wonder if the side-loaded snapshot version should be exactly the same as the current one, given that version bumping requires L1 interaction.
Also unclear if we should validate utxoToCommit and utxoToDecommit on the provided snapshot to match the last known state.
Concerned that a head can become stuck during a Recover or Decommit client input.
SideLoadSnapshot is the first ClientInput that contains a headId and must be verified when received by the node.
Uncertain whether WaitOnNotApplicableTx for localTxs not present in the side-loaded confirmed snapshot would trigger automatic re-submission.
I think this feature should not be added to TUI since it is not part of the core protocol or user journey.

2025-03-07

FT on SideLoad-Snapshot

Now that we have a head stuck on the initial snapshot, I want to explore how we can introspect the node state from the client side, as this will be necessary to create the side-load request.
Projecting the latest SnapshotConfirmed seems straightforward, but projecting the latest SeenSnapshot introduces code duplication in HeadLogic.aggregate and the ServerOutput projection.
These projections currently conflict heavily with pull#1860. For that reason, we are postponing these changes until it is merged.

2025-03-06

FT on SideLoad-Snapshot

We need to break down withHydraNode into several pieces to allow starting a node with incorrect ledger-protocol-params in its running configuration.
In this e2e scenario, we exercise a three party network where two nodes (node-1 and node-2) are healthy, and one (node-3) is misconfigured. In this setup, node-1 attemtps to submit a NewTx which is accepted by both healthy members but rejected by node-3. Then, when node-3 goes offline and comes back online using healthy pparams, it is expected to stop cooperating and cause the head to become stuck.
It seems that after node-3 comes back online, it only sees a PeerConnected message within 20s. Adding a delay for it to catch up does not help. From its logs, we don’t see messages for WaitOnNotApplicableTx, WaitOnSeenSnapshot, or DroppedFromQueue.
If node-3 tries to re-submit the same transaction, it is now accepted by node-3 but rejected by node-1 and node-2 due to ValueNotConservedUTxO (because it was already applied). Since node-3 is not the leader, we don’t see any new SnapshotRequested round being signed.
Node-1 and node-2 have already signed and observed each other signing for snapshot number 1, while node-3 has not seen anything. This means node-1 and node-2 are waiting for node-3 to sign in order to proceed. Now the head is stuck and won’t make any progress because node-3 has stopped cooperating.
New issue raised for head getting stuck issue#1773, which proposes to forcibly sync the snapshots of the hydra-nodes in order to align local ledger states.
Updating the sequence diagram for a head getting stuck using latest findings.
Now thinking how could we "Allow introspection of the current snapshot in a particular node"; as we want to be able to notice if the head has become stuck. We want to be able to observed who is missing to sign the current snapshot in flight (which is preventing from getting it confirmed).
Noticed that in onOpenNetworkReqTx we keep TransactionReceived even if not applicable, resulting in a list with potentially dupplicate elements (in case of resubmission).
Given that a head becoming stuck is an L2 issue due to network connectivity, I’m considering whether we could send more information about the local ledger state as part of PeerConnected to trigger auto-sync recovery based on discrepancies. Or perhaps we should broadcast InvalidTx instead?
Valid idea to explore after side-load.

2025-03-05

FT on SideLoad-Snapshot

Trying to reproduce a head becoming stuck in BehaviorSpec when a node starts with an invalid ledger.
Having oneMonth in BehaviorSpec's waitUntilMatch makes debugging harder. Reduced it to (6 * 24 * 3), allowing full output visibility.
After Bob reconnects using a valid ledger, we expected him to accept the transaction if re-submitted by him, but he rejects it instead.
It's uncertain whether Bob is rejecting the resubmission or something else, so I need to wait until all transactions are dropped from the queue.
Found that when Bob is resubmitting, he is in Idle state when he is expected to restart in Initial state.
This is interesting, as if a party suffers a disk error and loses persistence, side-loading may allow it to resume up to a certain point in time.
The idea is valid, but we should not accept a side-load when in Idle state—only when in Open state.
It seems this is the first time we attempt to restart a node in BehaviorSpec. Now checking if this is the right place or if I should design the scenario differently.
When trying to restart the node from existing sources, we noticed the need to use the hydrate function. This suggests we should not force reproducing this scenario in BehaviorSpec.
NodeSpec does not seem to be the right place either, as we don't have multiple peers connected to each other.
Trying to reproduce the scenario at the E2E level, now running on top of an etcd network.

2025-03-04

SB on fixing the persistence bug

Continuing where I left off yesterday - to fix a single test that should throw IncorrectAccessException but instead I saw yesterday:

 uncaught exception: IOException of type ResourceBusy

When I sprinkle some spy' to see the values of actually thread ids I don't get this exception anymore, just the test fails. So the exception is tightly coupled with how we check for threads in the PersistenceIncremental handle.
I tried labeling the threads and using throwTo from MonadFork but the result is the same.
Tried using withBinaryFile in both source and append and use conduit to stream from/to file but that didn't help.
Tried using bracket with openBinaryFile and then sink/source handle in the callback but the results are the same.
What is happening here?

2025-03-03

SB on api server as the event sink

There are only two problems left to solve here. First one being the IncorrectAccessException from persistence in the cluster tests. This one I have a plan on how to solve (have a way to register a thread that will append) and the other problem is some cluster tests fail since appropriate message was not observed.
One example test is persistence can load with empty commit.
I wanted to verify is the messages are coming through since the test fails at waitFor and I see the messages propagated (but I don't see HeadIsOpened twice!)
Looking at the messages the Greetings message does not contain correct HeadStatus anymore! There was a projection that made sure to update this feeld in Greetings message but now we shuffled things around and I don't think this projection works any more.
I see all messages correct (except headStatus in Greetings) but only propagated once (and we do restart the node in our test).
I see api server being spun up twice but second time I don't see message replay for some reason.
One funny thing is I see ChainRollback - perhaps something around this is broken?
I see one rebase mistake in Monitoring module that I reverted.
After some debugging I notice that the history loaded from the conduit is always empty list. This is the cause of our problems here!
Still digging around code to try and figure out what is happening. I see HeadOpened saved in persistence file and can't for the life of me figure out why it is not loaded on restart. I tried even passing in the complete intact event source conduit to make sure I am not consuming the conduit in the Server leaving it empty for the WSServer but this is not the problem I am having.
I remapped all projections to work with StateChanged instead of ServerOutput since it makes no sense to remap to ServerOutput just for that.
Suspecting that mapWhileC is the problem since it would stop each time it can't convert some StateEvent to ServerOutput from disk!
This was it - mapWhileC stops when it encounteres Nothing so it was not processing complete list of events! So happy to fix this.
Next is to tackle the IncorrectAccessException from persistence. I know why this happens (obviously we try to append from different thread) and sourcing the contents of a persistence file should not be guarded by correct thread id. In fact, we should allow all possible clients to accept (streamed) persistence contents and make sure to only append from one thread and that is the one in which hydra-node process is actually running.
I added another field to PersistenceIncremental called registerThread and it's sole purpose is to register a thread in which we run in - so that we are able to append (I also removed the check for thread id from source and moved it to append )
Ok, this was not the fix I was looking for. The registerThread is hidden in the persistence handle so if you don't have access to it from the outside how would you register a thread (for example in our tests).
I ended up registering a thread id on append if it doesn't exist and do a check if it is there but see one failure:


  test/Hydra/PersistenceSpec.hs:59:5:
  1) Hydra.Persistence.PersistenceIncremental it cannot load from a different thread once having started appending
       uncaught exception: IOException of type ResourceBusy
       /tmp/hydra-persistence-33802a411f862b7a/data: openBinaryFile: resource busy (file is locked)
       (after 1 test)
         []
         [String "WT",Null,Null]

I still need to investigate.

February 2025

2025-02-27

SB on state of things regarding api server memory consumption

There is no CommandFailed and ClientEffect
We don't have anymore GetUTxO client input therefore I had to call api using GET /snapshot/utxo request to obtain this information (in cluster tests)
For the tests that don't spin the api server I used TestHydraClient and it's queryState function to obtain the HeadState which in turn contains the head UTxO.
One important thing to note is that I had to add utxoToCommit in the snapshot projection in order to get the expected UTxO. This was a bug we had and nobody noticed.
We return Greetings and InvalidInput types from the api server without wrapping them into TimedServerOutput which is a bit annoying since now we need to double parse json values in tests. If the decoding fails for TimedServerOutput we try to parse just the ServerOutput.

Current problems:

After adding /?history=yes to hydra-cluster tests api client I started seeing IncorrectAccessException from the persistence. This is weird to me since all we do is read from the persistence event sink.
Querying the hydra node state in our Model tests to get the Head UTxO (instead of using GetUTxO client input) hangs sometimes and I don't see why. I suspect this has something to do with threads spawned in the model tests:

This is the diff, it looks benign:


 waitForUTxOToSpend ::
   forall m.
-  (MonadTimer m, MonadDelay m) =>
+  MonadDelay m =>
   UTxO ->
   CardanoSigningKey ->
   Value ->
   TestHydraClient Tx m ->
   m (Either UTxO (TxIn, TxOut CtxUTxO))
-waitForUTxOToSpend utxo key value node = go 100
+waitForUTxOToSpend utxo key value node = do
+  u <- headUTxO node
+  threadDelay 1
+  if u /= mempty
+    then case find matchPayment (UTxO.pairs u) of
+      Nothing -> pure $ Left utxo
+      Just (txIn, txOut) -> pure $ Right (txIn, txOut)
+    else pure $ Left utxo
  where
-  go :: Int -> m (Either UTxO (TxIn, TxOut CtxUTxO))
-  go = \case
-    0 ->
-      pure $ Left utxo
-    n -> do
-      node `send` Input.GetUTxO
-      threadDelay 5
-      timeout 10 (waitForNext node) >>= \case
-        Just (GetUTxOResponse _ u)
-          | u /= mempty ->
-              maybe
-                (go (n - 1))
-                (pure . Right)
-                (find matchPayment (UTxO.pairs u))
-        _ -> go (n - 1)
-
   matchPayment p@(_, txOut) =
     isOwned key p && value == txOutValue txOut

Model tests sometimes succeed but this is not good enough and we don't want anymore flaky tests.

2025-02-26

SN troubleshooting unclean restarts on etcd branch

Started by investigating hydra-cluster tests failing, for example this one erroring with

4) Test.EndToEnd, End-to-end on Cardano devnet, restarting nodes, close of an initial snapshot from re-initialized node is contested
    Process "hydra-node (2)" exited with failure code: 1
    Process stderr: RunServerException {ioException = Network.Socket.bind: resource busy (Address already in use), host = 0.0.0.0, port = 4002}

Seems like the hydra-node is not shutting down cleanly and scenarios like this
Isolated test scenarios where we simply expect withHydraNode to start/stop and restart within a certain time and not fail
Testing these tests on master it worked fine?! Seems to have something to do with etcd?
When debugging withHydraNode and trying to port it to typed-process, I noticed that we don't need the withHydraNode' variant really -> merged them
Back to the tests.. why are they failing while the hydra-node binary seems to behave just fine interactively?
With several threadDelay and prints all over the place I saw that the hydra-node spawns etcd as a sub-process, but when withProcess (any of its variants) results in stopProcess, the etcd child stays alive!
Issuing a ctrl+c on ghci has the etcd process log a signal detected and it shut downs
We are not sending SIGINT to the etcd process? Tried interruptProcessGroupOf in the Etcd module
My handlers (finally or bracket) are not called!? WTF moment
Found this issue which mentions that withProcess sends SIGTERM that is not handled by default
- Some familiar faces on this one
- This is also an interesting paragraph about how ctrl+c can be delegated to sub-process (not what we needed)
So the solution is two-fold:
- First, we need to make sure to send SIGINT to the etcd process whenever we are asked to shut down too (in the Etcd module)
- Also, we should initiate a graceful shutdown when the hydra-node receives SIGTERM
  - This is a better approach than making withHydraNode send a SIGINT to hydra-node
  - While that would work too, dealing SIGTERM in hydra-node is more generally useful
  - For example a docker stop sends SIGTERM to the main proces in a container

2025-02-19

SN working on etcd gprc client integration

When starting to use grapesy I had a conflict of ouroboros-network needing an older network than grapesy. Made me drop the ouroboros modules first.
Turns out we still depend transitively on the ouroboros-network packages (via cardano-api), but cabal resolver errors are even wors.
Adding a allower-newer: network still works
Is it fine to just use a newer version of network in the ouroboros-network?
The commits that bumped the upper bound does not indicate otherwise
Explicitly listed all packages in allow-newer and moved on with life

2025-02-17

SN working on etcd network connectivity

Working on PeerConnected (or an equivalent) for etcd network.
Changing the inbound type to Either Connectivity msg does not work well with the Authentication layer?
The composition using components (ADR7: https://hydra.family/head-protocol/adr/7) is quite complicated and only allows for a all-or-nothing interface out of a component without much support for optional parts.
In particular, an Etcd component that delivers Either Connectivity msg as inbound messages cannot be composed easily with the Authenticate component that verifes signatures of incoming messages (it would need to understand that this is an Either and only do it for Right msg).
Instead, I explore expanding NetworkCallback to not only deliver, but also provide an onConnectivity callback.
After designing a more composable onConnectivity handling, I wondered how the Etcd component would be determinig connectivity.
The etcdctl command line tool offers a member list which returns a list of members if on a majority cluster, e.g.

{"header":{"cluster<sub>id</sub>":8903038213291328342,"member<sub>id</sub>":1564273230663938083,"raft<sub>term</sub>":2},"members":\[{"ID":1564273230663938083,"name":"127.0.0.1:5001","peerURLs":\["<http://127.0.0.1:5001>"\],"clientURLs":\["<http://127.0.0.1:2379>"\]},{"ID":3728543818779710175,"name":"127.0.0.1:5002","peerURLs":\["<http://127.0.0.1:5002>"\],"clientURLs":\["<http://127.0.0.1:2380>"\]}\]}

But when invoked on a minority cluster it returns

  {"level":"warn","ts":"2025-02-17T22:49:48.211708+0100","logger":"etcd-client","caller":"[email protected]/retry<sub>interceptor</sub>.<go:63>","msg":"retrying
  of unary invoker
  failed","target":"etcd-endpoints://0xc000026000/127.0.0.1:2379","attempt":0,"error":"rpc
  error: code = DeadlineExceeded desc = context deadline exceeded"}
  Error: context deadline exceeded

When it cannot connect to an etcd instance it returns

  {"level":"warn","ts":"2025-02-17T22:49:32.583103+0100","logger":"etcd-client","caller":"[email protected]/retry<sub>interceptor</sub>.<go:63>","msg":"retrying
  of unary invoker
  failed","target":"etcd-endpoints://0xc0004b81e0/127.0.0.1:2379","attempt":0,"error":"rpc
  error: code = DeadlineExceeded desc = latest balancer error: last
  connection error: connection error: desc = \\transport: Error while
  dialing: dial tcp 127.0.0.1:2379: connect: connection refused\\"}
  Error: context deadline exceeded

When implementing pollMembers, suddenly the waitMessages was not blocked anymore?
While a litte crude, polling member list works nicely to get a full list of members (if we are connected to the majority cluster).
All this will change when we switch to a proper grpc client anyways

2025-02-05

SB on running conduit only once for projections

Current problem we want to solve is instead of passing a conduit to mkProjection function and running it inside we would like to stream data to all of the projections we have.
Seems like this is easier said than done since we also rely on a projection result which is a Projection handle that is used to update the TVar inside.
I thought it might be a good idea to alter mkProjection and make it run in ConduitT so it can receive events and propagate them further and then, in the end return the Projection handle.
I made changes to the mkProjection that compile

mkProjection ::
-  (MonadSTM m, MonadUnliftIO m) =>
+  MonadSTM m =>
   model ->
   -- | Projection function
   (model -> event -> model) ->
-  ConduitT () event (ResourceT m) () -> 
-  m (Projection (STM m) event model)
-mkProjection startingModel project eventSource = do
-  tv <- newTVarIO startingModel
-  runConduitRes $
-    eventSource .| mapM_C (lift . atomically . update tv)
-  pure
+  ConduitT event (Projection (STM m) event model) m ()
+mkProjection startingModel project = do
+  tv <- lift $ newTVarIO startingModel
+  meventSource <- await
+  _ <- case meventSource of
+    Nothing -> pure ()
+    Just eventSource ->
+      void $ yield eventSource .| mapM_C (atomically . update tv)
+  yield $
     Projection
       { getLatest = readTVar tv
       , update = update tv

but the main issue is that I can't get the results of all projections we need in the end that easy.

-- does not compile
headStatusP <- runConduitRes $ yield outputsC .| mkProjection Idle projectHeadStatus

We need to be able to process streamed data from disk and also output like 5 of these projections that do different things.
I discovered sequenceConduits which allows collection of the conduit result values.
Idea was to collect all projections which have the capability of receiving events as the conduit input.

[headStatusP] <- runConduit $ sequenceConduits [mkProjection Idle projectHeadStatus] >> sinkList

Oh, just realized sequenceConduits need to have exactly the same type so my plan just failed

I think I need to revisit our approach and start from scratch.

January 2025

2025-01-23

SB on state events streaming

So what we want to do is to reduce the memory footprint in hydra-node as the final outcome
There are couple of ADRs related to persisting stream of events and having different sinks that can read from the streams
Our API needs to become one of these event sinks
The first step is to prevent history output by default as history can grow pretty large and it is all kept in memory
We need to remove ServerOutput type and map all missing fields to StateChange type since that is what we will use to persist the changes to disk
I understand that we will keep existing projections but they will work on the StateChange type and each change will be forwarded to any existing sinks as the state changes over time
We already have PersistenceIncremental type that appends to disk, can we use similar handle? Most probably yes - but we need to pick the most performant function to write/read to/from disk.
Seems like we currently use eventPairFromPersistenceIncremental to setup event stream/sink. What we do is load all events from disk. We also have a TVar holding the event id. Ideally what we would like is to output every new event in our api server. I should take a look at our projections to see how we output individual messages.
Ok, yeah, projections are displaying the last message but looking at this code I am realizing how complex everything is. We should strive for simplicity here.
Another thought - would it help us to use Servant at least to separate the routing and handlers? I think it could help but otoh Servant can get crazy complex really fast.
So after looking at the relevant code and the issue https://github.com/cardano-scaling/hydra/issues/1618 I believe the most complex thing would be this Websocket needs to emit this information on new state changes. but even this is not hard I believe since we have control of what we need to do when setting up event source/sink pair.

SN on streaming events

Streaming events using conduit makes us buy into the unliftio and resourcet environment. Does this go well with our MonadThrow et al classes?
When using conduits in createHydraNode, the runConduitRes requires a MonadUnliftIO context. We have a IOSim usage of this though and its not clear if there can be a MonadUnliftIO (IOSim s) instance even?
We have not only loading [StateEvent] fully into memory, but also [ServerOutput].
Made mkProjection to take a conduit, but then we are running it for each (3 times). Should do something with fuseBoth or zip-like conduit combination.

2025-01-22

SN on multi version explorer

Started simplifying the hydra-explorer and wanted to get rid of all hydra-node, hydra-tx etc. dependencies because they include most of the cardano ecosystem. However, on the observer api we will need to refer to cardano-specifics like UTxO and some hydra entities like Party or HeadId. So a dependency onto hydra-tx is most likely needed.
Shouldn't these hydra specific types be in an actual hydra-api package? The hydra-tx or a future hydra-client could depend on that then.
When defining the observer API I was reaching for the OnChainTx data type as it has json instances and enumerates the things we need to observer. However, this would mean we need to depend on hydra-node in the hydra-explorer.
Could use the HeadObservation type, but that one is maybe a bit too low level and does not have JSON instances?
OnChainTx is really the level of detail we want (instantiated for cardano transactions, but not corrupted by cardano internal specifics)
Logging in the main entry point of Hydra.Explorer is depending on hydra-node anyways. We could be exploring something different to get rid of this? Got https://hackage.haskell.org/package/Blammo recommended to me.
Got everything to compile (with a cut-off hydra-chain-observer). Now I want to have an end-to-end integration test for hydra-explorer, that does not concern itself with individual observations, but rather that the (latest) hydra-chain-observer can be used with hydra-explorer. That, plus some (golden) testing agains the openapi schemas should be enough test coverage.
Modifying hydra and hydra-explorer repositories to integration test new http-based reporting.
- Doing so offline from a plane is a bit annoying as both nix or cabal would be pulling dependencies from the internet.
- Working around using an alias to the cabal built binary:

        alias hydra-chain-observer=../../hydra/dist-newstyle/build/x86_64-linux/ghc-9.6.6/hydra-chain-observer-0.19.0/x/hydra-chain-observer/build/hydra-chain-observer/hydra-chain-observer

cabal repl is not picking up the alias, maybe need to add it to PATH?
Adding a export PATH=<path to binary>:$PATH to .envrc is quite convenient
After connecting the two servers via a bounded queue, the test passes but sub-process are not gracefully stopped.

2025-01-21

SB on stake certificate registration

I created a relevant issue to track this new feature request to enable stake certificates on L2 ledger.
Didn't plan on working on this right away but wanted to explore a problem with PPViewHashesDontMatch when trying to submit a new tx on L2.
This happens both when obtaining the protocol-parameters from the hydra-node or if I query them from cardano-node (the latter is expected to fail on L2 since we reduce the fees to zero)
I added the line to print the protocol-parameters in our tx printer and it seems like changePParams is not setting the protocol-parameters correctly for whatever reason:

changePParams :: PParams (ShelleyLedgerEra Era) -> TxBodyContent BuildTx -> TxBodyContent BuildTx
changePParams pparams tx =
  tx{txProtocolParams = BuildTxWith $ Just $ LedgerProtocolParameters pparams}

There is setTxProtocolParams I should probably use instead.
No luck, how come this didn't work? I don't see why setting the protocol-parameters like this doesn't work....
I even compared the protocol-parameters loaded into the hydra-node and the ones I get back from hitting the hydra-node api and they are the same as expected
Running out of ideas

2025-01-20

SB on looking at withdraw zero problem

I want to know why I get mismatch between pparams on L2?
It is because we start the hydra-node in a separate temp directory from the test driver so I got rid of the problem by querying hydra-node to obtain L2 protocol-parameters
The weird issue I get is that the budget is overspent and it seems bumping the ExecutionUnits doesn't help at all.
When pretty-printing the L2 tx I noticed that cpu and memory for cert redeemer are both zero so that must be the source of culprit
Adding separately cert redeemer fixed the issue but I am now back to PPViewHashesDontMatch.
Not sure why this happens since I am doing a query to obtain hydra-node protocol parameters and using those to construct the transaction.
Note that even if I don't change protocol-parameters the error is the same
This whole chunk of work is to register a script address as a stake certificate and I still need to try to withdraw zero after this is working.
One thing I wanted to do is to use the dummy script as the provided Data in the Cert Redeemers - is this even possible?

2025-01-08

SN on aiken pinning & cleanup

When trying to align aiken version in our repository with what is generated into plutus.json, I encountered errors in hydra-tx tests even with the same aiken version as claimed.
Error: Expected the B constructor but got a different one
Seems to originate from plutus-core when it tries to run the builtin unBData on data that is not a B (bytestring)
The full error in hydra-tx tests actually includes what it tried to unBData: Caused by: unBData (Constr 0 [ Constr 0 [ List [ Constr 0 [ Constr 0 [ B #7db6c8edf4227f62e1233880981eb1d4d89c14c3c92b63b2e130ede21c128c61 , I 21 ] , Constr 0 [ Constr 0 [ Constr 0 [ B #b0e9c25d9abdfc5867b9c0879b66aa60abbc7722ed56f833a3e2ad94 ] , Constr 1 [] ] , Map [(B #, Map [(B #, I 231)])] , Constr 0 [] , Constr 1 [] ] ] , Constr 0 .... This looks a lot like a script context. Maybe something off with validator arguments?
How can I inspect the uplc of an aiken script?
It must be the "compile-time" parameter of the initial script, which expects the commit script hash. If we use that unapplied on the transaction, the script context trips the validator code.
How was the initialValidatorScript used on master such that these tests / usages pass?
Ahh .. someone applied the commit script parameter and stored the resulting script in the plutus.json! Most likely using aiken blueprint apply -v initial and then passing the aiken blueprint hash -v commit into that.
Realized that the plutus.json blueprint would have said that a script has parameters.

Logbook 2025 H1 - cardano-scaling/hydra GitHub Wiki

December 2025

2025-12

2025-12-01

SB on partial fanout

November 2025

2025-11-26

SB on debugging close transaction

SB on partial fanout

2025-11-13

SB on debugging hydra-node memory issues

2025-11-04

SB on Partial Fanout

October 2025

2025-10-06

Some notes on fanout limit

2025-10-01

SB on deposit reference script

September 2025

2025-09-30

SB on deposit reference script

2025-09-29

SB on deposit reference script

2025-09-24

SB on Best practise on committing user scripts

2025-09-23

SB on Best practise on committing user scripts

2025-09-22

SB on Blockfrost smoke tests

2025-09-16

SB on H4 in Increment tx

2025-09-08

SB on partial tokens output

2025-09-03

SB on partial tokens output

2025-09-03

SB on partial tokens output

August 2025

2025-08-14

SB on debugging H4 user issue on deposit

July 2025

2025-07-22

SB on fixing the input queue bug

2025-07-03

SB on increment bug

June 2025

2025-06-19

SB on debugging user logs

May 2025

2025-05-19

SB on looking at the user issue with committing

2025-05-13

SB on hydra-node using blockfrost

2025-05-12

SB on hydra-node using blockfrost

2025-05-08

SB on hydra-node using blockfrost

2025-05-07

SB on hydra-node using blockfrost

2025-05-05

SB on hydra-node using blockfrost

April 2025

2025-04-30

SN on fixing deposits

SB on hydra-node using blockfrost

2025-04-29

SB on hydra-node using blockfrost

2025-04-28

SN on fixing deposits using behavior tests

2025-04-26

SN refocusing on deposit fixes

2025-04-24

SN on model based testing interlude

2025-04-23

SN on fixing deposits

2025-04-22

SN on fixing deposits

2025-04-09

SB on blockfrost wallet queries

SB on finalizing recover/decrement observations should not be conditional

SN on exploring `dingo`

⚠️ GitHub.com Fallback ⚠️