Dojo
Paymail and UTXO Management Server
- Open-ended Paymail server
- Benford’s Law and On-chain Privacy Out-of-the-box
- Identifies users by their XPUB keys
- Flexibility to create any Bitcoin transactions of your choice
- Full support for K-chains and R-puzzles
- MetaNet friendly
- Ninja keeps keys client-side, while Dojo tracks UTXOs online
Disclaimer
This is alpha-stage software. Things could be horribly broken at this stage and you could lose money or have payments dropped, routed to the wrong place or you could inadvertantly send your money into cyberspace where it will never be found.
Peer-to-peer Privacy Systems Research, LLC (the people who create this software) disclaim all liabilities, express and implied, surrounding the API at this stage.
Revenue and Money Model
The service is free right now. In the future (as we write tests, get feedback, harden the API against vulnerabilities and go through the other growing pains) we will charge a processing fee based on what makes sense. This will probably be an additional output script in the requestOutputsForP2PTransaction, or maybe an extra change output sent back with createOutgoingTransaction. Either way, it will be a fee paid by the sender (obviously), so the receiver will need to choose how to handle this.
Perhaps we can also work with receivers (our users) to let them pay month-to-month in order to waive this fee. This will be the Dojo / UTXONinja revenue model.
Initial Setup
This Dojo can be configured for use with its sister client library, the UTXO Ninja. Refer to the appropriate documentation for how to use that library. This server's Dojo URL is below:
https://dojo.babbage.systems
Paymails and Domains
This server is currently configured to allow Paymail registration under the following internet domain names:
- gateway.cash
- babbage.systems
- tyweb.us
Primary Paymail Domain: gateway.cash will be used by default when a new xpub key is registered.
API Errors
Errors are sent back with appropriate HTTP status codes. The status field of the response object will be "error" in these cases. The object will also contain a machine-readable code field and a human-readable description field. See the Table of Errors for a complete list of error codes.
Table of Errors
- ERR_PAYMAIL_NOT_FOUND
- ERR_INTERNAL
- ERR_UNAUTHORIZED
- ERR_INVALID_REFERENCE
- ERR_INVALID_TRANSACTION_STATUS
- ERR_REQUEST_EXPIRED
- ERR_PAYMAIL_MISMATCH
- ERR_TRANSACTION_REJECTED
- ERR_SENDER_SIGNATURE_CHECK
- ERR_PAYMAIL_UNAVAILABLE
- ERR_INVALID_PAYMAIL_DOMAIN
- ERR_NOT_SUFFICIENT_FUNDS
- ERR_BAD_REQUEST
- ERR_INVALID_OUTPOINT
Notes on ERR_NOT_SUFFICIENT_FUNDS
When you use the createOutgoingTransaction endpoint to attempt to request inputs and change outputs to a transaction that would require more funds than are available in the current context (you are trying to spend more Bitcoins than you have on hand), the server will generate ERR_NOT_SUFFICIENT_FUNDS (an "NSF Error.")
In addition to the error code and a human-readable description field, the error object will also contain totalSatoshisNeeded and moreSatoshisNeeded fields. These values are calculated based on the cost of only the required outputs and whatever inputs were available to be selected. They do not necessarily reflect the actual amount of satoshis you need to have on hand, because we do not account for the cost of miner fees associated with redeeming and spending the new output(s) in the transaction.
Depending on the application, it can be better to over-shoot these estimates by a little and have change outputs. Factor in your current receive policy, fee rates, the types of outputs used and others if you need to be precise.
API Request Authentication
To manage UTXOs and Paymail handles, you will authenticate to the server with an xpriv key. A new user is created the first time a properly-signed request comes in from an xpub key. A new, random Paymail handle is generated.
Authentication Object
To authenticate requests, provide an auth object in the POST request body. It must contain the following fields:
xpub
This is the extended public key in BIP32 serialization format that serves to identify the user, derive their output scripts and sign requests.
dt
This is a timestamp in seconds
signature
This is the cryptographic signature made over the data in the request. An example of the data to sign is given below:
Sign data in the following format with the xpriv key:
{"body":{"bar":"baz"},"dt":1606590241,"url":"http://localhost:3998/foo","xpub":"xpub661MyMwAqRbcFuw54zk5G6hbrEyFNa2tcD5JVm11NkC7DYYcdtuVQYoj5RSpRcG9Xt7NSCctmYXLFdzvL3249KbEWy8S3RgCemH9zFLff6R"}
The following JavaScript code should work for generating valid request signatures. The order of the fields when creating the signature (body, dt, url, xpub) is important! The complete API endpoint URL (not just the relative path) should be used. The dt must be within 2 minutes of the server clock.
const bsv = require('bsv') // bsv1 const hdPriv = bsv.HDPrivateKey.fromString('xprv...') const privKey = hdPriv.privateKey const xpub = hdPriv.hdPublicKey.toString() const dt = parseInt(Date.now() / 1000) const payload = { paymail: 'foo@bar.com', name: 'John Smith', photoURL: 'https://foo.com/baz.png' } const messageToSign = JSON.stringify({ body: payload, dt, url: 'https://foo.com/changePaymail', xpub }) const signature = bsv.crypto.ECDSA.sign( bsv.crypto.Hash.sha256(Buffer.from(messageToSign)), privKey ).toString() const completedBody = { auth: { xpub, dt, signature }, ...payload }
Receive Policies
The receive policies are how you will define the types, amounts and derivation paths for incoming transactions destined for your Paymail handle(s). The receive policy has the below fields:
- derivationPathForPaymailIdentityKeys: The XPUB derivation path to use when allocating new Paymail identity keys. If m/0/1 is provided, the first key is allocated at m/0/1/0 and subsequent keys are allocated at m/0/1/n, where n is the index of the key. Currently, only one Paymail identity key is used. By default, m/0/1 is used.
- derivationPathForP2PKHUTXOs: The XPUB derivation path to use when allocating new P2PKH output scripts. If m/0/2 is provided, the first key is allocated at m/0/2/0 and subsequent keys are allocated at m/0/2/n, where n is the index of the key. By default, m/0/2 is used.
- derivationPathForP2RPHUTXOs: The XPUB derivation path to use when allocating new P2RPH output scripts. If m/0/3 is provided, the first key is allocated at m/0/3/0 and subsequent keys are allocated at m/0/3/n, where n is the index of the key. By default, m/0/3 is used.
- numberOfDesiredP2PKHUTXOs: Specify the number of P2PKH UTXOs that the server will attempt to keep on-hand at any given time. (By default, 6)
- numberOfDesiredP2RPHUTXOs: Specify the number of P2RPH UTXOs that the server will attempt to keep on-hand at any given time. (By default, 6, but note that P2RPH is also disabled by default)
- minimumDesiredP2PKHUTXOValue: The minimum desired value for P2PKH UTXOs given in satoshis. However, there are situations where this value cannot be upheld, such as when the total value of an incoming payment is too low. (By default, 75000)
- minimumDesiredP2RPHUTXOValue: The minimum desired value for P2RPH UTXOs given in satoshis. However, there are situations where this value cannot be upheld, such as when the total value of an incoming payment is too low. (By default, 50000)
- acceptP2PKHUTXOs: Whether to accept and enable P2PKH outputs (default true)
- acceptP2RPHUTXOs: Whether to accept and enable P2RPH outputs (default false)
It should be noted that the policies surrounding the minimum desired value of UTXOs will take precedence over the policies surrounding the number of UTXOs to be kept on hand.
Endpoint Documentation
get https://dojo.babbage.systems/.netlify/functions/dojo-serverless/verifyPublicKeyForPaymail/:paymail/:pubkey
Use this endpoint to verify that a given Paymail identity public key is owned by a Paymail handle being hosted by this server.
Parameters
- paymail: Substitute the :paymail placeholder in the GET URL with the actual paymail handle you want to look up.
- pubkey: Substitute the :pubkey placeholder in the GET URL with the actual public key you want to look up.
Example Response
{ "handle": "foo@bar.com", "pubkey": "032fc032bd3465c3870c8bd0582b934ff3803e176819c7f80693dcfafdf83b539c" }
get https://dojo.babbage.systems/.netlify/functions/dojo-serverless/getAvatarForPaymail/:paymail
Use this endpoint to obtain the avatar (name and profile photo URL) for a given Paymail alias which is hosted on this server. The photo URL may be an HTTP or UHRP-compliant URL.
Parameters
- paymail: Substitute the :paymail placeholder in the GET URL with the actual paymail handle you want to look up.
Example Response
{ "name": "John Smith", "photoURL": "https://foo.com/photo.png" }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/requestOutputsForP2PTransaction/:paymail
Use this endpoint to request outputs that a Paymail user will accept for payment. Provide the number of satoshis you want to request, then use the specified outputs in a BSV transaction that you will submit along with the reference number to the submitP2PTransaction endpoint for validation and broadcast.
Parameters
- paymail: Substitute the :paymail placeholder in the POST URL with the actual paymail handle you want to look up.
- satoshis: Specify the number of satoshis in the JSON body of the POST request.
Example Response
{ "outputs": [ { "script": "76a914f32281faa74e2ac037493f7a3cd317e8f5e9673688ac", "satoshis": 10000 }, { "script": "76a914f32281faa74e2a455555567a3cd317e8f5e9673688ac", "satoshis": 45557 } ], "reference": "fasodjfadsfasdfas=" }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/submitP2PTransaction/:paymail
After requesting payment outputs with the requestOutputsForP2PTransaction function, use this endpoint to submit the signed transaction hex for a transaction that contains the specified outputs. Also include the reference number you received with the output scripts and amounts. This endpoint will broadcast the transaction to the Bitcoin network.
Parameters
- paymail: Substitute the :paymail placeholder in the POST URL with the same paymail handle you specified earlier.
- hex: Provide the signed raw transaction in hexadecimal format within the JSON body of the POST request.
- reference: Also provide the reference value that you received when you requested output scripts for the transaction.
- metadata.sender: Provide the Paymail handle for the person who sent the transaction (Optional).
- metadata.pubkey: If a sender is specified, also provide the Paymail identity public key that they used to sign the request.
- metadata.signature: If a sender is specified, provide a signature over the fields of this request that are defined in the appropriate Paymail specifications.
- metadata.note: You can also give the recipient of this transaction a note that wmay be displayed in their UI.
Example Response
{ "txid": "123...", "note": "The transaction has been received." }
get https://dojo.babbage.systems/.netlify/functions/dojo-serverless/getIdentityKeyForPaymail/:paymail
Use this endpoint to obtain the identity key associated with a Paymail that is hosted on this server.
Parameters
- paymail: Substitute the :paymail placeholder in the GET URL with the actual paymail handle you want to look up.
Example Response
{ "bsvalias": "1.0", "handle": "john@doe.com", "pubkey": "032fc032bd3465c3870c8bd0582b934ff3803e176819c7f80693dcfafdf83b539c" }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/getCurrentPaymails
Use this endpoint to obtain a list of the Paymails associated with your xpub key. In the current version of this API, only one Paymail is supported per user. In subsequent versions, we may add support for multiple aliases, depending on user and developer feedback.
Parameters
- auth: The authentication object (see the authentication section)
Example Response
{ "aliases": [ "who@john.galt" ] }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/changePaymail
Use this endpoint to change your Paymail handle. Currently, you need to have at least one unspent output registered with the Dojo in order to change your Paymail.
Parameters
- auth: The authentication object (see the authentication section)
- paymail: The new Paymail that will be associated with your xpub key
Example Response
{ "newPaymail": "joe@example.com" }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/getCurrentReceivePolicy
Use this endpoint to obtain the current receive policy for your xpub key. See the section on receive policies for information about how to create the best receive policy for your application's needs.
Parameters
- auth: The authentication object (see the authentication section)
Example Response
{ "policy": { "...": "..." } }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/setReceivePolicy
Use this endpoint to update the receive policy associated with your xpub key. Refer to the receive policy section for more information on the required fields in the policy object.
Parameters
- auth: The authentication object (see the authentication section)
- policy: The receive policy (see the policy section)
Example Response
{ "status": "success" }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/getTransactionOutputs
Use this endpoint to obtain a list of transaction outputs managed by this server.
Parameters
- auth: The authentication object (see the authentication section)
- spendable: If given as true or false, only outputs that have or have not (respectively) been spent will be returned. If not given, both spent and unspent outputs will be returned. (Optional).
- type: If provided, only outputs of the specified type will be returned. If not provided, outputs of all types will be returned. (Optional).
- limit: Provide a limit on the number of outputs that will be returned. (Optional).
- offset: Provide an offset into the output list from which to begin returning outputs. (Optional).
Example Response
[ { "txid": "123...", "vout": 5, "amount": 314159, "outputScript": "76...", "type": "P2PKH", "spendable": true } ]
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/getTransactions
Use this endpoint to obtain a list of transactions associated with your xpub key.
Parameters
- auth: The authentication object (see the authentication section)
- limit: The maximum number of transactions to return at one time. (Optional).
- offset: The offset into the list of transactions from which results will be returned. (Optional).
- involving: If provided, only transactions involving the specified Paymail handle will be returned. (Optional).
- orderBy: If provided, specifies the sort order of the list. Valid options are "newest-first" and "oldest-first". By default, the order is "oldest-first". (Optional).
Example Response
[ { "txid": "txid1", "status": "completed", "amount": 314159, "senderPaymail": "foo@bar.com", "recipientPaymail": "me@example.com", "note": "Here is money.", "isOutgoing": true, "created_at": "2020-11-28T18:25:26.104Z" } ]
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/getTotalOfUnspentOutputs
Use this endpoint to obtain the total value of the unspent transaction outputs. Currently, the total value is always returned, but in the future, if we decide to implement output baskets, transaction labels and other sorting schemes, it will be possible to provide more specificity.
Parameters
- auth: The authentication object (see the authentication section)
Example Response
{ "total": 45568 }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/createOutgoingTransaction
Use this endpoint to create an outgoing transaction and generate change outputs. Provide script lengths and amounts for each of the outputs you want in the transaction, the desired transaction fee and whether you need to select R-puzzle (P2RPH) inputs. You will receive the selected inputs that you can sign and the change outputs to use (if any).
Parameters
- auth: The authentication object (see the authentication section)
- requiredOutputs: An array of objects, each of which represents one of the outputs you will include when you create and sign the transaction.
- requiredOutputs.scriptLength: The length of the output script in bytes
- requiredOutputs.amount: The amount of satoshis in the output (can be zero for data outputs)
- rPuzzleNeeded: Whether you want to include at least one P2RPH R-puzle input that you will be free to sign with any key when unlocking. Note that by default, P2RPH is disabled and this will cause an error if you have no P2RPH inputs available to spent. (Optional)
- feePerKb: The desired transaction fee in satoshis per KB (other fee models may be added in the future). By default, it will be 550. It can be 0 for a no-fee transaction. (Optional)
- foreignInputs: This provides a way for you to specify external (non-Dojo) inputs for your transaction. The field is an array of input objects, and if this field is provided it will disable the Dojo input selection logic. The selected inputs to your transaction will always be all of the foreign inputs. If you provide an empty array of required outputs, all of the foreign inputs will be spent to change, which provides a way for you to "fund" the xpriv key.
- foreignInputs[element].amount: The number of satoshis in the foreign input
- foreignInputs[element].outputScript: The nexadecimal locking script for the input
- foreignInputs[element].txid: The txid of the transaction
- foreignInputs[element].vout: The output index of the output in the transaction
- foreignInputs[element].type: The output type ("P2PKH" or "P2RPH")
Example Response
{ "selectedInputs": [ { "txid": "ffaa...", "vout": 5, "derivationPath": "m/0/3/4", "type": "P2RPH", "lockingScript": "006a44...", "amount": 192168 }, { "txid": "ccdd...", "vout": 2, "derivationPath": "m/0/2/15", "type": "P2PKH", "lockingScript": "88ac...", "amount": 255255 } ], "changeOutputs": [ { "script": "442266889955", "satoshis": 4488 } ] }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/processOutgoingTransaction
After you have created an outgoing transaction with the createOutgoingTransaction endpoint, after you have signed the new transaction, after it has been submitted to the recipient and accepted by the Bitcoin network into the mempool, use this endpoint to process the transaction, thereby marking the change outputs as spendable and spending the selected inputs.
Parameters
- auth: The authentication object (see the authentication section)
- reference: The reference that you received along with your selected inputs and change outputs from createOutgoingTransaction
- hex: The raw transaction hex for the transaction that contains the selected inputs and change outputs.
- recipient: The Paymail handle of the recipient of this transaction. (Optional).
- note: A note about this transaction. (Optional).
Example Response
{ "note": "The transaction has been processed and broadcasted." }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/setNameAndPhotoURL
Use this endpoint to set a name and profile photo URL that will be resolved when people request an avatar for the Paymail handle.
Parameters
- auth: The authentication object (see the authentication section)
- name: The name that you would like to use
- photoURL: The UHRP or HTTPS URL to a photo that you would like to use as your avatar
- paymail: The Paymail handle you want to associate with the name and photo URL
Example Response
{ "status": "success" }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/updateTransactionStatus
Use this endpoint to update the status of a transaction. This is useful for flagging incomplete transactions as aborted or reverting a completed transaction back into a pending status if it never got confirmed.
Parameters
- auth: The authentication object (see the authentication section)
- reference: The reference number associated with the transaction
- status: The new desired status for the transaction
Example Response
{ "status": "success" }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/updateOutpointStatus
Use this endpoint to update the status of one of your outputs, given as the TXID of a transaction and the vout (output index) in that transaction. This is useful for flagging transaction outpoints as spent if they were inadvertantly broadcasted or used without properly submitting them to the Dojo, or to undo the spending of an output if it was never actually spent.
Parameters
- auth: The authentication object (see the authentication section)
- txid: The txid of the transaction that created (not spent!) the output which you would like to update
- vout: The index of the output in the transaction
- spendable: A boolean indicating the corrected outpoint spendability status. If true, this output can once again be selected spent in subsequent transactions. If false, the output will be treated as if it has already been spent and will not be used or selected again.
Example Response
{ "status": "success" }
post https://dojo.babbage.systems/.netlify/functions/dojo-serverless/migrate
This is an administrative endpoint used by the server administrator to run any new database migrations and bring the database schema up-to-date.
Parameters
- migrateKey: The database migration key that was configured by the server administrator.
Example Response
{ "status": "success" }
Tips and Tricks
As you build, you may find the tips and tricks from this section useful when architecting the best applications and services.
Dealing With "Phantom UTXOs"
Sometimes, Dojo-managed UTXOs get inadvertently spent without properly going through the transaction creation or processing steps. There are two ways to deal with this type of situation, depending on the circumstances.
1. Just use processOutgoingTransaction: works when createOutgoingTransaction was used (currently unproven)
If, for example, you created a transaction with createOutgoingTransaction, sent it to a merchant who broadcasted it and accepted it as payment, but before you could use processOutgoingTransaction to mark your UTXOs as spent and activate your change outputs, your internet connection dropped, then the Dojo does not know that the transaction was ever broadcasted.
It may try to spend the same UTXOs again, causing for future transactions to be rejected with missing inputs. To resolve this, when you come back online (or when you otherwise learn of the error), you could find the reference number for the missing transaction with getTransactions, then use a service provider to obtain the raw hex and subsequently process it as normal. If you would rather not do this, the next section describes a more straightforward (though somewhat less ideal) method.
2. Manually mark UTXOs as spent: works no matter what
If, for example, you loaded your XPUB key into another Bitcoin wallet and sent transactions using the traditional non-P2P way (do not do this!), the transactions were not created in accordance with Dojo's instructions, and createOutgoingTransaction was never even used. You don't have a reference number, so you can't just process them with processOutgoingTransaction.
You can recover from this by using the updateOutpointStatus endpoint, manually flagging the afflicted UTXOs as spent. That way, Dojo won't try to spend them again. If you are in a situation where you now have rogue change outputs created by other software, if you want to bring them back under Dojo management here is how:
Just use createOutgoingTransaction while specifying all of the rogue UTXOs as foreign inputs. Specify no required outputs, thereby sending all of the foreign inputs to change (and back under Dojo management). Sign the transaction and process it with processOutgoingTransaction to get your UTXOs back into the system.
Work With Me
I am looking for someone to help maintain, test, stabilize and improve the Dojo and Ninja projects. If you want to help out with making this into production-ready software, I will sign a contract and give you a percentage of fees earned by the API in exchange for keeping it maintained or something.
the-code-of-competence-is-the-only-system-of-morality-on-a-gold-standard@gateway.cash