Zero-fee Layer 2 planet sales

by ~sitful-hatred, published on

Similar content available at networked subject_

Updated June '22 with v3

Introduction

The release of layer 2 rollups for Azimuth means planets can be spawned very cheaply – so cheaply, in fact, that Tlon is picking up the tab. If you operate an L2 star, spawning planets means generating a CSV of planet codes, each of which can be redeemed within Bridge, at no cost to you or the recipient.

That isn't the only change to Bridge – planet codes are also new. Think of it as an invitation code; a short string of text that can be redeemed and used to automatically kick off a process to convert a code into cryptographic property.

In combination, these two changes mean that buying or selling a planet is no more difficult than selling a password. This opens the door to using traditional ecommerce tools, without requiring specialized techniques like atomic swaps, or manually processing transfers. This post is about doing just that – together, we will set up a self-hosted, automated planet sales platform, using open source software and Bitcoin payments. All you need is a Linux server, an operational star, and a domain name – this stack can be hosted on a spare computer in your house.

The stack

The heart of this assembly is a self-hosted crypto payment platform called BTCPay Server. I have nothing but good things to say about this tool; it is a fully-featured web store platform that you host yourself, which automatically handles invoicing, and is connected to a Bitcoin full node (also handled automatically). All of the hard stuff is taken care of by the Docker image – the installation experience is very close to plug & play. You can create a store and receive payments without going through a payment processor, or even renting a server.

We will create a store using BTCPay, and configure it to call a webhook upon successful payment, which will call the BTCPay API. The API will deliver the customer's email address to the rest of the script, which will reference and update your planet code list, and fire off an email to both of you via SendGrid.

Prep & Installation

Email setup

As I mentioned, you'll need a domain name to set everything up. Since we'll also be making use of email triggers, it would be convenient to use the same domain for email as our server. If you don't already have your domain configured for email, it's as simple as editing a few DNS entries and paying a few dollars a month to a provider.

We will also register a SendGrid account. SendGrid will allow us to programmatically send emails using nice templates and custom variables. You are allowed 100 outbound emails a day using a free account – if you need to start sending more than this, it's likely you can afford the $15/month. After you've created the account, verify a single sender address, or validate the domain using CNAME records. Go ahead and create an API key and save it somewhere for later.

You'll also need your domain or a subdomain pointed at the IP address of your server you'll be using. If you want to host it on your home network, point an exposed reverse proxy at the IP of the device you'll be using (i.e., set up port forwarding for 80/443 to the device running the reverse proxy). Caddy makes this very simple, and automatically handles TLS certificates to secure your connections; your configuration file might look something like the first entry here:

btcpay.mydomain.com {
    reverse_proxy 192.168.0.25:80
}
urbit.mydomain.com {
    reverse_proxy localhost:8080
}

You can set up a reverse proxy on a gateway device, or on the server you'll be using itself. I already had an existing web gateway set up on my home network, so I just added another entry for this domain.

BTCPay installation

Open a command line on the server you'll be using. Prepare to install BTCPay by cloning the repo as root:

$> sudo su -
$> mkdir BTCPayServer
$> cd BTCPayServer
$> git clone https://github.com/btcpayserver/btcpayserver-docker
$> cd btcpayserver-docker

We'll be using the Docker installation method, so we'll need to set some environmental variables for the install script before we begin. Enter each of these in sequence, after modifying the first one with the domain name you'll be using:

$> export BTCPAY_HOST="btcpay.domainname.com"
$> export NBITCOIN_NETWORK="mainnet"
$> export BTCPAYGEN_CRYPTO1="btc"
$> export BTCPAYGEN_ADDITIONAL_FRAGMENTS="opt-save-storage-xs"
$> export BTCPAYGEN_REVERSEPROXY="nginx"
$> export BTCPAYGEN_LIGHTNING=""
$> export BTCPAY_ENABLE_SSH=true
$> export REVERSEPROXY_DEFAULT_HOST="$BTCPAY_HOST"
$> export BTCPAYGEN_EXCLUDE_FRAGMENTS="$nginx-https"

Two notes: the last line can be left out if you're not using a different reverse proxy to handle HTTPS; and you can remove the 'opt-save-storage' argument if you don't want to prune your full node's blockchain (and you have ~500GB of disk space to spare). Now we're ready to run the installation script:

$> ./btcpay-setup.sh -i

Everything should be handled automatically. Try opening the docker0 interface's IP address or the domain you assigned in your browser – you should be met with a BTCPay splash screen and login option. Go ahead and create an account with a very strong password. The first account created will automatically receive admin privileges.

BTCPay configuration

Once logged in, you'll see a floating box at the bottom right corner with information about your blockchain sync progress. This might take a day or two to complete, but we can continue setting everything up in the meantime. You can check in on the progress periodically.

Take a look at all the options in the 'Server settings' menu. Configure the 'Email server' tab using the server smtp.sendgrid.net, port 587, username apikey, and the API key you generated earlier as the password. Check off the 'Enable SSL' box and hit save.

Next, click the 'Stores' menu button at the top, and create a new store. Here you can set up prices and billing preferences, as well as theming your public pages. You will also be able to import an xPub Bitcoin wallet here. I use Wasabi Wallet, which conveniently allows you to export your wallet in a format that BTCPay can import. This allows you to manage your funds without exposing your keys to the payment server, on top of procedurally generating new addresses for every transaction. It may also be possible to import your Urbit's master ticket xPub, though my first attempt wasn't successful. After your store is created and configured, go to 'Apps' and create a point-of-sale app. This is basically your storefront, where you'll create a listing for planet sales. You can denominate prices in Bitcoin or USD, depending on your preference.

There is plenty more to tweak and configure in BTCPay, particularly creating stylesheets for your shop. Once your shop is ready, retrieve the URL for the POS app you created. You can embed this as an iframe on an existing site, or link to it directly from elsewhere. You can also make it the root path of the domain you're using, so that someone visiting btcpay.mydomain.com is dropped directly into the shop.

Connecting to BtcPay

Once BTCPay and is installed and configured and you have a store (separate instructions), go to Settings > Access Tokens, and generate a legacy API key.

Copy the base64-encoded version of your API key:

Use this to set your BTCPAY_API_KEY variable in webhookMailer.sh.

Next, select your store's 'app' from the left-hand menu in BTCPay, scroll down, and expand Notification URL Callbacks. Enter the address of your webhook (e.g. http://172.17.0.1:9000/hooks/emailer).

Webhook installation

Open a command line on the server running your Docker instance. We're going to install the prerequisites first:

$> sudo apt install jq curl webhook sqlite3

Now clone the webhook repo:

$> git clone https://github.com/yapishu/WebhookEmailer
$> cd WebhookEmailer

Inside you will find two main components: webhookMailer.sh, a shell script, and emailer.json, a webhook configuration. There is also a CSV of test data, and a webhook systemd unit we'll install momentarily.

Webhooks are like a little listener running on a server that are configured to trigger an event when they receive a message. The event can be pretty much anything you want – in our case, it will be running webhookMailer.sh, using the email address that the customer provided.

You will need to personalize a few variables inside of settings.data – open it in a text editor:

FROM_EMAIL="example@test.com"
FROM_NAME="Arvo Ames"
# The name of the CSV the script will import
CSV_FILE="planets.csv"
SENDGRID_API_KEY="SG._________"
# Base64 Legacy API key from store settings > access tokens
BTCPAY_API_KEY="_____________________________"
# URL your webhook host can reach your BTCPayServer at
BTCPAY_API_URL="http://localhost:80"
# https://mc.sendgrid.com/dynamic-templates
SG_TEMPLATE="d-___________"

CSV_FILE is the name of the spreadsheet of planets & codes that you download from Bridge, placed in the same directory as the script. When triggered, this script will import the CSV into a database, look there for the first row with 2 entries, use the contents to email the customer, then append their email address and a timestamp to that database row so that it will be skipped the next time it is run. Note that there are no accommodations for running out of rows! BTCPay allows you to set a number of items in inventory, and I recommend lining it up with the number of available planets in your database (more details below).

A couple of variables are for sending emails through SendGrid – enter the API key from earlier, the email address validated, and a template ID number. You can create a SendGrid template here – I recommend using a free template editor to create a nice looking one. Insert the variables {{planet-code}} and {{planet-name}} in the body of your email, and the email template will automatically insert data from your CSV in their place.

Go ahead and open emailer.json in a text editor. This is the configuration file for the webhook, which you'll need to modify. Fix the path to point at your WebhookEmailer directory (the directory it's sitting in). This is the only required fix, but webhook has configuration parameters that allow you to whitelist IP addresses. This is a good idea if your webhook is accessible via the internet. You can see an example of how to enable this option here.

Finally, let's install this webhook as a systemd module so we don't have to manually start it. First, edit emailer.service in a text editor, fix the path in the emailer.json file so that it points at your WebhookEmailer directory, and change the User and Group parameters to whatever your username is on this server. Now for the installation:

$> sudo cp emailer.service /etc/systemd/system/emailer.service
$> sudo systemctl enable emailer
$> sudo systemctl start emailer

That's it – everything should be humming along and plugged in together. You can test it by creating an item in your store that costs $0 and submitting your email address with an order. Note that it will fail to send if the recipient is the same as the 'from' address you set in the script, since it is bcc'd onto the emails.

Notes about webhookMailer.sh

The main script, webhookMailer.sh, is a monolithic bash script that will make API calls to Sendgrid, as well as maintain a database of planet codes that it generates from CSV exports from Bridge.

In order to make use of it, you will need to deposit your CSV in the same directory as the script, with a filename that corresponds with the variable CSV_FILE -- I use Planets.csv. When the script executes, it will immediately look for a matching CSV that has not been marked as imported, and create/add it to a Sqlite3 database.

The script will then proceed to find the first unused code in the DB, extract the name, code and claim URL, and pass them to the Sendgrid API via curl. Then it will mark the planet as sold by adding a recipient email and timestamp to that row. It will also make a check to see whether there is only one planet left in the database, and if so, will send you an email.

Note that this script will also automatically dedeuplicate entries in the database; you do not need to manually remove any entries for planets codes that have not been claimed yet, as long as they have been imported previously. This is necessary because unclaimed codes will continue to show up in your Bridge CSV exports until somebody claims them.

Included in the repo is a script named getStatus.sh -- this will make a few queries to the database file and print statistics, like how many codes it contaians and how many are unused. You can also run this command to dump the full contents to the terminal:

sqlite3 db.sq3 'SELECT * FROM planets;'

When you generate another batch of codes, simply rename it and move it to your webhook script directory. On the next run it will be imported to the database.

In the course of putting together this stack I also found a very neat tool called Litestream, which will automatically send streaming backups of your Sqlite database to an S3 bucket. You may want to make use of it, in addition to traditional, less frequent backup scripts for the rest of the stack.

Wrap-up

Consider this an MVP release – this stack works, but there is plenty of room for improvement and expansion if you want to tinker. For instance, we have configured this solely for Bitcoin, but Lightning Network and many other coins are supported by BTCPay if you install the relevant plugin. You could probably get BTCPay to work with the Urbit-Bitcoin full node stack instead of letting it run its own full node, or possibly on an Umbrel. In principle, you could also modify the webhook script to support fiat payments.

You can find a basic action log for the shell script in Transactions.log in your WebhookEmailer directory.


More tutorials and documentation available the networked subject_ blog, or in the on-network group: web+urbitgraph://group/~matwet/networked-subject


Comments