Debugging Tips
This is a living document and will be updated as the ecosystem matures & grows.
If you have a tip you'd like to share with others, please open a PR to add it here!
itest - Investigating Flaky Tests
We developed a tool called itest to help with debugging flaky tests. It runs
the same test iteratively
itest Usage
Run the following command to see the usage of itest:
./tools/scripts/itest.sh
itest Example
The following is an example of itest in action to run the TxClient_SignAndBroadcast_Succeeds
test in the pkg/client/tx 50 times in total (5 consecutive tests over 10 runs).
make itest 5 10 ./pkg/client/tx/... -- -run TxClient_SignAndBroadcast_Succeeds
poktrolld query tx - Investigating Failed Transactions
tl;dr Submitted Transaction != Committed Transaction
After a transaction (e.g. staking a new service) is successfully sent to an RPC node, we have to wait until the next block, when a proposer will try to commit to the network's state, to see if its valid. If the transaction's (TX) state transition is invalid, it will not be committed.
In other words, receiving a transaction (TX) hash from the poktrolld CLI doesn't mean it was committed.
However, the transaction (TX) hash can be used to investigate the failed transaction.
poktrolld query tx Example
The following is an example of poktrolld query tx in action to investigate a failed transaction.
In this example, the command to add a new service is executed as follows, returning the TX hash shown.
However, the service does not appear in the list of services when querying the full node.
poktrolld tx service add-service "svc1" "service1" 1 --from $SUPPLIER_ADDRESS --chain-id=poktroll
The TX hash is returned by the above command:
txhash: 9E4CA2B72FCD6F74C771A5B2289CEACED30C2717ABEA4330E12543D3714D322B
To investigate this issue, the following command is used to get the details of the transaction:
poktrolld query tx \
--type=hash 9E4CA2B72FCD6F74C771A5B2289CEACED30C2717ABEA4330E12543D3714D322B \
--node https://shannon-testnet-grove-seed-rpc.poktroll.com
Which shows the following log entry:
info: ""
logs: []
raw_log: 'failed to execute message; message index: 0: account has 100000 uPOKT, but
the service fee is 1000000000 uPOKT: not enough funds to add service'
The output above shows the cause of the transaction failure: insufficient funds. Fixing this by adding
more funds to the corresponding supplier account will allow the transaction to result in the expected
state transition.
If you are reading this and the 9E4CA... hash is no longer valid, we may have done a re-genesis of
TestNet at this point. Please consider updating with a new one!
poktrolld query tx supports an --output flag which can have the values text or json. This can be useful for programatic querying or in combination with tools like jq, e.g.:
poktrolld query tx \
--type=hash 9E4CA2B72FCD6F74C771A5B2289CEACED30C2717ABEA4330E12543D3714D322B \
--node https://shannon-testnet-grove-seed-rpc.poktroll.com \
--output json | jq .raw_log
The above command will produce the following output:
"failed to execute message; message index: 0: account has 100000 uPOKT, but the service fee is 1000000000 uPOKT: not enough funds to add service"