Substratum Dev Team Show & Tell — 5-July-19 (i35)

Show & Tell weekly sessions are an excellent insight into the challenges and triumphs facing the Substratum development team. For those following these Show & Tell discussions, the format of the progress breakdown has just gotten slightly more tense and exciting, because the team is down to the last handful of critical items (cards) to move to the first Release Candidate (RC1) of the SubstratumNode


As always, B.J. Allmon heads the meeting by outlining the Substratum Node Version 1 Strategic Goals:

  • Priority 1. Make the Internet Free (as in freedom) and Fair. — “As a SubstratumNode user I can allocate my spare computing resources so that the Internet can be a free and fair place for the entire world.”
  • Priority 2. Earn Cryptocurrency. Change the world. — “As a SubstratumNode user I earn $SUB as content is routed through my node so that I am incentivized to make the Internet free and fair.”


BJ gave a bit more clarity here about how the Node vision is being developed in real usages:

In Priority 2, ‘content’ is a big word — it could mean a lot of things. It could be routing services, exit services or even blockchain services eventually. This could be where people don’t have access to a GETH miner or a blockchain service URL (like Infura), they can get the block information from users on the Substratum network.

Many of the Tactical Focus Items listed have been done for a while. Most of these have been done for the purpose of RC1 and the others that are Work-In-Progress will be built-out further after RC1 to provide more in-depth features.

GUI updates have notably been challenging and involve a lot of work.


The ‘Show and Tell’ segment

Still plugging way, and we are looking at the data differently from here on out, with the remaining cards. — B.J. Allmon

The team review the completed work for the week, including Telling the story of each card, or Showing the completed feature.


Setup DNS Utility to require rustfmt

This was actually completed for all the sub-projects within the repository, as well as the DNS Utility. Basically just reduces noise in the teamPersonal Repos.

There is a known bug in the DNS Utility that needs to be fixed for Windows, so this will assist in cleaning that up (tech debt)

Prevent fraudulent consuming wallet usage — Steve Swing

When your node builds a route, it will take the network public keys of the route nodes, and provide the proof of having the private key for the consuming wallet being used to pay for the services — this produces a signature. This lets all the nodes on the route know that the requesting node has:

  1. Provided a wallet for payment
  2. Can sign transactions for that wallet

This will prevent node users consuming on the network without a valid wallet to pay for services.

This does create an incompatible version with users who have downloaded the v0.4.8 binaries, since this updated code is not in that downloaded version.

Another main function of this completed card, is that the smart contract of the token is also called into the signature, so testnet won’t be compatible with mainnet.

Implement clippy linting for dns_utility:

This was a tech debt item

Clippy provides scans of the code, and provide hints that you are running certain code functions.

SHOW — Wallet Password Unlock for the GUI — Steve Swing:

BJ put the SubNode GUI on the screen.

To understand the wallet functionality a bit more, its best to clarify how wallets and wallet passphrases are derived (not just specific to Node wallet generation)

Passphrases and passwords are two separate things.

The mnemonic passphrase is the additional word you include with the BIP39 mnemonic words, that creates a more secure wallet. (usually it’s a 13th word or 25th word when generating new wallet seed words)

The words are used to generate a seed value.

The seed value can be combined with a wallet derivation path (n/44’/60’/0/0 etc) and this produces a key pair (wallet public address and private key)

To keep that seed secure, a password can also be collected to encrypt it.

The mnemonic seed value is stored encrypted in the database, but the password is used to decrypt it and provide the wallet info (mnemonic seed) to the Node itself.


Essentially all of this is in place so Node can pay bills for routing!

The GUI on screen showed the prompt that the user sees to provide the Password to unlock the wallet being used for consuming.


BJ explained that the format of tracking progress has changed.


From this point on, velocity averages will not help predict the final time frame to complete the RC1. BJ said he will tweet updates as the remaining cards for RC1 are completed.

2 cards are in flight and are really close — they unlock two cards in the To-Do bucket. One of the four remaining is also dependent on another being completed first.

Therefore RC1 completion cannot be predicted based on how fast the team previously were completing cards, since they are down to the final few and they are codependent on each-other.

Thankfully the work on the routing for RC1 has already been completed.

Dan Wiebe explained more on how the cards became more technical resulting in several of the final ones remaining:

We assumed that every time you started Node, you wanted to use the same earning and consuming wallets and could store and reference them in the database. However, it was discovered that changing the consuming wallet can cause issues with tables stored in the database.

So with this challenge in mind, the work relating to the consuming wallet was split into several cards.

BJ clarified there is still the ability to have several nodes running that are sharing the same earning wallet, but there is some complexity to work around, as the accountant module will be tracking all the transactions relating to the same earning wallet across many nodes.

It’s definitely something that needs to be thoroughly tested in the testnets.



Team Capacity Utilization overall initiative looks great!

This week the team had 2 cards that were critical RC1 cards, and two others that were tech debt that relate to Version 1, but not necessarily RC1 — that was played to keep several of the development team busy working on constructive items while others play out the RC1 cards.


For those who want more technical insight, I tried to capture some of the Guest Open Space chatter. If you don’t want to ‘geek out’, you can stop reading here!


Guest Questions/Comments during meeting (paraphrased!)

Aussie_Crypto — Question from the community was do you have plans to have testers try to attack Node on a clandestine and monetisation level

BJ & Steve — Yes definitely. What’s challenging is that many of the High-Level Security Execs we deal with are not experienced in hacking in this area.

Andreas — Offering a bug bounty would be a great way for this — community could be incentivized for this.

Brian — Since the latest commit has breaking changes, the community isn’t able to continue testing. So do we anticipate a v0.4.9 version so the community can test a pre-cut binary so the testing velocity can be continued?

BJ — its a good question — we will have to see how we sit mid-week

Steve — it would be good to see another minor release, as the next few cards will have compatibility changes


Disclaimer: I am not an official Substratum team member — I am a community moderator. This write-up is interpretation of the Show & Tell meeting and data released from the team. None of this is to be taken or construed as investment-related or as financial advice.

Substratum Dev Team Show & Tell — 28-June-19 (i34)

Show & Tell weekly sessions are an excellent insight into the challenges and triumphs facing the Substratum development team. Each week a set of slides is released and a group conference is held with dialogue and examples of work planned, completed and lurking on the horizon!


This week has a small number of items on the Show & Tell, but they are very important.

As always, B.J. Allmon heads the meeting by outlining the Strategic Goals of v1.0.0 Substratum Node:

Priority 1 — Make the Internet Free (as in Freedom) and Fair

Priority 2 — Earn Cryptocurrency. Change the World

Tactical Focus Items are outlined listed and briefly mentioned as Work-In-Progress


Many of the listed items are completed for Release Candidate 1, and this is very promising in terms of having a secure and stable candidate to fully test, leading to the final Version 1 (v1.0.0) release!

BJ explained that the team also has spent a little time mapping out CrytoPay flows, and also integration ideas in the Amplify product suite and Amplify wallet.


The ‘Show and Tell’ segment this week shows only a small number of cards, since there are very few cards left to work on for RC1.

But the things in it are GOOD! — B. J. Allmon


Outlined below are the items to report this week:

TELL — Proxy Client timeout charges for 0-byte — Dan Wiebe

Actually fixed now!

Dan spoke a little about this last week. The issue arises when a connection or stream is broken between a browser and a server. If the browser closes or stops requesting the connection to a website for example, SubNode was not communicating that to the website server (including if this happens from the server end also). This arises because SubNode sits in between the browser and the server in the network layer. So there needed to be a way to send the “Drop-Report” across the network to the exit node to inform the Server to terminate the connection. And vice-versa for the server to report to the originating node browser if that connection drops or ceases to connect.

Tell — Write logs with closures and macros, not functions — Dan Wiebe

When SubNode writes logs, there is usually a string written which is input into a log function. The level of the logging determines whether the logging subsystem outputs the log text or not.

In many cases the work involved with the logging was not required, and performance and resources were being wasted doing this.

So instead, the function and string work will be avoided if the logging is not to be written when your logging level is set very high (to WARN or ERROR)

So this actually boosts performance when you don’t have the logging level set to DEBUG or TRACE

SHOW — Release Notes for v0.4.8 — B.J. Allmon

B.J. Allmon went over the release notes and fixes that were implemented in the recent minor release.

The main reason this minor release was pushed out, was so the fixes could apply to testers using the GUI, so they can also have the benefit of the performance fix with the accountant thread.

SHOW — Make logs consistent — Dan Wiebe

Logging was pulled up on screen, and the old versus the new were compared in text form.

Overall, there is now less logging being written that was wasting space and creating extra parsing. This includes referencing the libraries being used that get output into the logs.

So the logs are more streamlined but also consistent over the different log level outputs.


Incredible to see the 1-week figure on the trajectory for RC1

Incredible to see the 1-week figure on the trajectory!

But it can be misleading even though things are looking good.

First time in the history of my software development career, I’ve seen this same 2-week forecast over a 2 month range! — B.J. Allmon

B.J. Allmon explained that 4th July is a USA holiday where they aren’t working and the talk in the community has been it would be great for excitement and marketing etc.

But the team is unemotional about it, so they are just trying to get the work done.

In the To-Do bucket, there are just 3 cards, but only 2 can be worked on in parallel — the third are dependant on the other ones, so it’s an order-dependant process to complete them all.


Team Capacity Utilization

B.J Allmon added an ‘Ideal’ benchmark bar, and the ‘Initiative’ bar, which shows across the majority of the work done on the SubNode initiative.

The recent bar showing ‘Other’ related to the work on the logging, which wasn’t critical to RC1, but allowed members of the team to be productive on other enhancement.

The team has four things in flight which are really close.

Stay tuned for another Show & Tell next week with the Substratum Developers — hopefully its an announcement of the Release Candidate 1 being ready for testing in the real world!


For those who want more technical insight, I tried to capture the Guest Open Space chatter. If you don’t want to ‘geek out’, you can stop reading here!


Guest Questions/Comments during meeting (paraphrased!)

Andreas reminded of the changing (dynamic) IP concern

BJ — Some ideas are being discussed, which will be exciting to see how it can be implemented. The team agrees it is totally doable.

Steve — we need to investigate what happens to the IP on the ISP end, and how it affects the nodes connected. It would require another neighbor on the network to ‘notice’ that the IP address changes, and then be able to reintroduce the node into the network again.

Cuba — the feedback in the community is that the performance has improved!

BJ — has seen a lot of the State error coming up in the test community and lots of items noticed with Windows 10. The team is getting to that, and a lot of work for Windows users is in the backlog and will be worked on for that.

BJ — we have considered the test community suggestion to have some official nodes running to start the network. But we wouldn’t want to ship the software with nodes already embedded in the GUI

Currently the overall UX experience is the ‘Version1 experience.’

We have a lot of ideas — based on things like Proxy, instead of subverting the machine DNS.

The GUI needs to be completely revamped at some point.

We need to rethink that, but now the priority is a functional product that proves that it works.

Brian — the ‘Consume’ button seems to be flawed

Especially in the Windows GUI, as even though the DNS entry is showing up on the IPv4 settings, routing stops working when a user clicks CONSUME button from the SERVING state.

This can be fixed with a work-around by having the State as SERVING, open up the IPv4 properties in your network settings, and set the DNS manually to 127.0.0.1 — works every time so far.

BJ — this seems to be an issue and the fix is in the works


Disclaimer: I am not an official Substratum team member — I am a community moderator. This write-up is interpretation of the Show & Tell meeting and data released from the team. None of this is to be taken or construed as investment-related or as financial advice.

Substratum Dev Team Show & Tell — 21-June-19 (i33)

Apologies to the community for missing last week’s Show & Tell Summary

Show & Tell weekly sessions are an excellent insight into the challenges and triumphs facing the Substratum development team. Each week a set of slides is released and a group conference is held with dialogue and examples of work completed, tabled and planned on the horizon!

This week has a handful of exciting fixes, the most important being an extreme performance boost, showing very fast network speeds compared to previous code base!


Abram Cookson the CTO, begins the meeting in B.J’s absence. As always, the meeting beings by outlining the Strategic Goals of v1.0.0 Substratum Node:

Priority 1 — Make the Internet Free (as in Freedom) and Fair

Priority 2 — Earn Cryptocurrency. Change the World

Tactical Focus Items are outlined listed and briefly mentioned as Work-In-Progress



The ‘Show and Tell’ segment this week explores not one, but SIX achievements!

The team was noticeably in great spirits, and were all excited to explain various parts of the work completed for the week.

It was also very nice to hear from a few other team members this week, who had really shown amazing persistence and progress with their cards.


Outlined below are the major items to digest this week!

Tell — Proxy Client Timeout charges for 0-byte

This refers to when a failed request doesn’t make it past the proxy client to the server, the route is not charged to the user (basically the request drops on the floor and is not charged for sending bytes relating to the request itself).

Substratum Base Version Number

This allows the Node software to know if other Node users are using a different version, which can help compatibility as the network and software develops in future. This version number will be in the payload in CORES packets so software can identify this.


Node GUI IP address field shows HTML content when 503 error — Justin Hohman

The GUI has been tweaked to allow more usability. When Node is activated in the GUI, the setting prompts for information as before, but won’t stall if info needs to be changed. The “Save” button now is “Stop & Save”, which lets you change your configuration or neighbor descriptor without freezing the GUI.

When you change the State from Serving to Consuming and vice-versa, it saves your External IP details and neighbor descriptor.

A bug was also removed that leaked HTML error output text into the descriptor field

SHOW — Node performance ‘not good enough’ for v1 — Justin Hohman

Developer Justin Hohman showed a demo on-screen of a 7-node cluster hosted in Google Cloud, being created with TNT (Test Net Tools). Before using this test Sub Network, speed is normally 150Mb/s on their Wifi network.

On the test network demo’d live, the speed test shows 58 Mb/s, which is impressive considering the team were running the live Zoom meeting at same time.

Before, our community testers were only getting close to 10Mb on cloud instances!

To create this massive performance boost, the accountant process was running together with the other processes with node. By separating the accountant process into its own thread, this removed database calls, and can run on its own without blocking the rest of what is going on in the Node software


SHOW — 10th Node Error — Justin Hohman

Justin spun up another 10 node network, (which was cool as they have an Easter Egg that names each node after a Pokemon!), and showed a visual graph of all the nodes connected successfully to each other.

There was a major bug that was causing issues when a 10th node was trying to connect to a single descriptor.

Now the testing shows this fix has prevented issues with the node network scaling out from a single node descriptor!

SHOW — Support HD Wallet addresses — Steve Swing

In a previous Show & Tell, it was demonstrated that a wallet mnemonic seed can be created using Node (the 12 words that generate public wallet addresses using BIP32). Now the Node software knows how to generate the mnemonic phrase and use this to derive both Consuming and Earning wallets addresses, for use immediately within the Node.

The consuming wallet will need a private key, as this is used to sign the transactions when you consume on the network.

Using the command line, Steve showed this process, and also how to encrypt the wallet with an additional password.

Basically, earning and consuming wallets can be created, encrypted and stored within the Node from inside the software, making this process streamlined for users who want to start fresh with new wallets to hold and spend their SUB tokens from running Node!



Its quite amazing how the trajectory is steady, with target of only 2 weeks for completion of Node v1.0.0 Release Candidate 1!

Abram spoke so proudly of the team and how they went full steam ahead, pumping out the completed work this week like a well oiled machine.



“6.99 cards completed” — Abram Cookson

6 cards were completed (almost seven!) this week, with 3 being bug cards and 3 planned cards.

Solid utilization with the team being equally productive on both bug cards and planned work/feature cards.


Overall it was an amazing week! Not only did the team achieve some important fixes that were hindering the Node network expansion, but also a critical performance fix that now helps users see great speeds across the network in testing. Other comments from them hinted at even further performance tweaks that will see even more boosts.

Stay tuned for another Show & Tell next week with the Substratum Developers!

For those who want more technical insight, I tried to capture most of the Guest Open Space chatter that was really interesting. If you don’t want to ‘geek out’, you can stop reading here!



Guest Questions/Comments during meeting (paraphrased!):

Cuba had an emergency BBQ — he gently reminded us which is rich baritone voice that he only gets to enjoy 3 days of sun a year where he lives!

Guest: Is next week the ‘payout week’ for node?

Steve: Payout card is being worked on now

We have the current ability to detect payouts, but not blockchain confirmations right now.

Hope that it will be completed quickly.

Crypt0_Kiwi: Is 10th node bug truly fixed? In Brian’s honour can we see a demo of 10 nodes?


Justin spun up 25 nodes immediately

Showed the visual digraph of the nodes — all nodes were connected!

Further testing needed with Node dropping old and rejoining etc. As opposed to all nodes being started at same time.

Cuba: big shout out to the testing group for all their work!

Team: Agree! Internal Testing group praised!

Dan Wiebe: Preach brother!

Abram Cookson: I feel like the ‘feedback loop’ is the best out of any company I have been part of.

Maybe because of the open source nature, but it’s quite awesome, unlike how it can be in a corporate environment or conference

T: How are you dealing with wallet addresses?

Only ones created within node can be used?

Steve: for now we are just dealing with mnemonic seed.

We have a RC2 card — so you can recover a wallet

We have designed node in order of what a user would use with running the software.

ETH addresses are compatible with Ropsten and mainnet, so we assumed users wouldn’t be using their real SUB and SUB wallets as the version roll-out.

In the next few updates, users should be able to import wallets with a json key store.

It may also be possible for users using MEW type wallets and not hardware wallets.

We will have a way to do this with CLI or a config setting in the .toml file, and use password to decrypt it

Another input to be developed would be to recover mnemonic phrase (same as the one on HD wallet), to initialize the wallet and derive the wallet within node using the passphrase (13th word)

Crypt0_Kiwi: If a large HTML download fails, will the partial request have a zero charge?

Dan: we are charging by CORES package. So large files are not packed into one cores packet

If a large file of 10gb aborted after 2gb for example, the CORES packets were routed and the other nodes routing have no idea that things failed.

So they are charging for routing.

At moment, no — if someone routes or transmits to you, you get charged for it.

Steve: bytes are charged as transferred, if browser doesn’t save it then that’s that. ISP is the same with data caps. If your computer drops the data after it has been recieved, ISP still counts it.

Question from Microoo on Telegram:

What happens when originate node makes a request of downloading a huge file and then cancels or disconnects? Do hoppers and exit node get notified of the cancellation?

Dan — working on that card now with Kevin.

If one end of connection gets closed, the originating node will translate that closure into a CORES package and transmit it to the exit node, informing it to close this stream on the transmitting end. So it will interrupt the data stream.

So the books should be kept in sync in terms of payments for routing and requests sent.

Kevin brought up another large node cluster and showed it on visual digraph

Same test but did daisy chain of connecting nodes (one connected to each one in sequence as they join)

Bridge bearer structure looks good. No errors could be seen.

Previous demo in the Show was all the nodes connecting to the same single node on startup.

Marcel: Wanted brief explanation of the problem with the 10th node failure

Kevin: Issue was that once you got to 10 nodes, the new guy was being introduced kind-of in a chain.

It would debut to 1st node who says “Well I’m full, but here is this other guy.”

The other guy would connect and say well you need another neighbour, then in a chain this kept continuing.

But the Node was connected, but did not inform the others that they were connected. So introduction gossip kept happening

When passes were happening, that situation wasn’t being detected.

Crypt0_Kiwi: It seems that nodes with 5 neighbors are rare to see in the new visuals

Abram: It’s like a rare pokemon! X-D

Get your masterball out!


Disclaimer: I am not an official Substratum team member — I am a community moderator. This write-up is interpretation of the Show & Tell meeting and data released from the team. None of this is to be taken or construed as investment-related or as financial advice.

Substratum Dev Team Show & Tell —7-Jun-19

Show & Tell weekly sessions are an excellent insight into the challenges and triumphs facing the Substratum development team. Each week a set of slides is released and a group conference is held with dialogue and examples of work completed, tabled and planned on the horizon!

This week revealed completed work on an exciting critical component for monetization — the integration of wallets securely into Node (with mnemonic phrases)


B.J. Allmon lead developer begins the meeting, as always, outlining the Strategic Goals of v1.0.0 Substratum Node:

Priority 1 — Make the Internet Free (as in Freedom) and Fair

Priority 2 — Earn Cryptocurrency. Change the World

Tactical Focus Items are outlined listed and briefly mentioned as Work-In-Progress


B.J. mentioned specifically that work has just started on node encryption in the last week or so, which is extremely important for a secure testnet and ultimately the first Release Candidate (RC1).

Also clarified at this point, was that before a ‘Version 1’ (v1) is finally released, team and community have to run the Release Candidates on testnets and identify things that are not on the planned scope, including defects, bugs and hardening.

Storymapping for Substratum Host has just begun, which is reliant on Node to even be possible.


Normally the ‘Show and Tell’ segment takes up the bulk of each weekly meeting, but this is the sparsest slide we have seen since these meetings have been broadcast!

The mnemonic phrase item actually took several weeks to finally finish — which is basically integrating how wallets will be connected into Node itself, both for earning and consuming by users.

But this goes to show how difficult the items are right now that need to be completed.

Outlined below are the two major items to digest this week!


Stop sending useless intros via Gossip — Dan Wiebe:

This card was identified by testers last week, and was moved forward because the issue itself could create further problems in future node-network building. When Node itself sends intro or passing Gossip (basically the function that says — “Hey Node B — contact this new Node C!”), the code has not been checking whether the new node was already known to the others.

This completed card now provisions that there should never be a node that has less than 2 neighbors (unless there is a true network problem)

This wasn’t prioritized for RC1, but since the testing community saw the issues this was creating, the team prioritized this to fix this.

SHOW — Generate BIP39 master seed demo — Steve Swing:

Steve demonstrated on screen the new functions integrated with blockchain wallets.

This can be considered one of the most critical and exciting components as this is the true core of monetization! — without it there is no way to earn and pay SUB tokens!

This card dealt with coding a number of command line (CLI) parameters, and allowing users to integrate wallets into the Node software.

The component is designed to allow both Hardware (think Ledger or Trezor) and Software based wallets (think Metamask or online MyEtherWallet) to be used while running a Node.

The code itself allows a BIP39 mnemonic phrase to be input, or to even generate a new mnemonic phrase from within the code (which is used to produce unique public wallet addresses).

The Testnet (Ropsten on ETH) will default to a 12 word-count for the mnemonic phrases, so it compatible with wallets like Metamask.

Mainnet will default to 24 word phrases.



“We knew early it would be a tough week.” — BJ

For the first time in 10 weeks, the forecasted weeks counter did not reduce.

It remains at 4 weeks.

The remaining cards are very tough and are complex — but the challenge also relates to the few number of cards left to work with.

Again this is a moving target!



All work was on the 2 ‘Planned’ cards this week, hence the 100% utilization in green.

There has been talk around the community about why the team seems focused on building all the components on the command line.

B.J. reinforced that these features are the underpinning functionality, so the GUI can even be built. So it has to be strategic, and have core features in place for Node on the system code side.

The team still very much cares about user friendliness and GUI development!

Now, the UI features can be built up including creating and selecting an earning/consuming wallet, user settings and more.


Overall it was an interesting week. B.J. did sound a little frustrated about week countdown remaining the same, but simultaneously proud that this has not happened in 10 weeks with the development! The critical component of the wallet integration, and parallel development with the blockchain services and referencing, really shows that monetization is being fleshed out successfully. It sounds like in the coming week a minor release could be expected, along with more work on encryption and blockchain integration.

Stay tuned for another week with the Substratum Developers!

For those who want more technical insight, I tried to capture some of the Guest Open Space chatter that was really interesting. If you don’t want to ‘geek out’, you can stop reading here!


If you don’t want to ‘geek out’, you can stop reading here!

Guest Questions/Comments during meeting (paraphrased!):

Steve: We have cards so that the GUI has the interface to prompt and recover the mnemonic phrases and generate them.

If you feel comfortable with your current wallet, you can import your recovery phrase you initialized your HD wallet with, into the node software and keep your SUB where it is to use in Node.

We (Node or dev team) don’t save your mnemonic phrase — We encrypt the resulting mnemonic seed produced by the phrase and your mnemonic passphrase (25th word if you have this)

Right now its fake encryption, so don’t use a real wallet yet if you are testing.

We will have an area where your initialized wallet is displayed in the future GUI

Brian: Couple things — great job on the critical fixes!

Have you upgraded to rust 1.35?

Steve: I think we are upgraded for the current builds

Brian: Are Port specifications in the UI?

The use-case is quite important, as users don’t want to open all the range of ports and use DMZ.

If I launch the UI, and it picks a port outside the range, it may not be able to do a full handshake into the network.

Port needs to be opened before initializing node

(so it’s almost a chicken vs egg dilemma)

Dan: Router should be able to be fully closed, and the entire handshake should work after you open the required port. If that is not the case, then we need to test for that.

Steve: if you are on a new machine that’s never used Node before, you can provide the clandestine port parameter in CLI, configure the script to use open port

Dan: it’s not in the UI now. But what might help you if you have used Node on CLI, it will have chosen a random port and stored it in the .db file. It will still use that port in future initializing every time Node is run. (unless you delete the database)

Brian: the challenge is we are rebuilding things frequently and having to wipe the .db file frequently when testing

BJ: no card yet for UI port assignment, but we will create one

Justin (developer): HTTPS traffic was not flowing through node. Routing was becoming sporadic. It was pulling different exit nodes — right now some HTTPS won’t work for TLS sessions (handshake). Team is fixing that, but it is a major challenge right now

BJ: Thing developers have noticed the most with current coding — ‘the fix is easy’ but ‘testing the fix’ is extremely difficult.

BJ made a NEW WORD! ‘Craftspersonship’ ?

Brian: Relating to HTTPS/TLS issues, thank you to Dan for writing up an explanation on how this should be working.

The current panics in testing community now are placeholder as some features aren’t implemented yet.

BJ: Panics are a problem, but we need to fix it. One is the blockchain url parameter not being specified.

Dan: Some issues still remain when blockchain url parameter isn’t provided. Even if you are serving-only, Node still will have to access blockchain services to verify blockchain info on your behalf. Blockchain info may be censored in your area, so the node network is leveraged to provide block info for payment verification etc.

NOTE: this is actually an aspect where users can earn more if their node is needed to fetch blockchain service data for other nodes who cannot access it, haven’t set it up in Node, or are restricted from those services in their country!

This is an aspect where your node may need to consume even though it is set to serving-only the network.

Steve: we need to explain — if you are serving-only and don’t have access to the blockchain data, you are trusting that other nodes have SUB to pay the services you are providing — if you can’t access blockchain, then you cannot verify if people can pay!

We don’t want that problem to exist! So we are writing the app so Node will have access one way or another.

If you don’t supply the URL it will check in time intervals due to blockchain API — we will design so it won’t crash/panic the node.

Andreas: A small request regarding public IPs — can Node recognize a node that may have a new or updated public IP, after it has already connected to the network?

Dan: Technically, the way Node is written now, it may not be necessary for Public IP to be a parameter. But we haven’t gotten around to addressing the fact that dynamic IP addresses will change and Node won’t adapt to this at this time.

Andreas: One idea to make it simpler — one solution some software have is to provide a DNS Entry instead of a static IP, so if we can just provide a DNS name for us, it can maybe be updated. But would need to write the code to update the database on the network

Steve: remember that the SQL database is not storing any data, just who and what you owe.

Dan: it also stores the port tables of delinquency, who owes what, and only stores wallet address. Not private keys.

Now stored in memory — there is hashmap of neighbourhood directory, IP addresses, neighbour lists, earning wallet addresses and a bunch of stuff about nodes, also that’s where we make the routes from.

When node goes down, that all goes away. Its ephemeral.

BJ: it was very intentional. All data we handle, we want to handle it with care. We want it to go away if a computer is stolen etc. What data we store is more ‘safe’ for storage from a safety perspective.

BJ: one thing tester noticed also, is that there is a performance boost by log filtering, and using log_level info instead of debug.


Disclaimer: I am not an official Substratum team member, just a community supporter. This write-up is interpretation of the Show & Tell meeting and data released from the team. None of this is to be taken or construed as investment-related or as financial advice.

Substratum Dev Team Show & Tell — 31-May-19

Show & Tell weekly sessions are an excellent insight into the challenges and triumphs facing the Substratum development team. Each week a set of slides is released and a group conference is held with dialogue and examples of work completed, tabled and planned on the horizon!

This week showed some exciting troubleshooting, and some major steps forward in both the monetization and neighborhood dynamics areas of Substratum Node.



With an air of battle hardened precision, B.J. Allmon lead developer begins the meeting by always outlining the Strategic Goals of v1.0.0 Substratum Node:

Priority 1 — Make the Internet Free (as in Freedom) and Fair

Priority 2 — Earn Cryptocurrency. Change the World

Tactical Focus Items are also listed and briefly mentioned as Work-In-Progress

Notably mentioned this week was how the team worked with Brian M, an amazing community node-running tester, who live tested a 5-node cluster with the developer team to troubleshoot. This led to 3 defect cards (the team uses Agile lean project management techniques) outlining fixes.

The team created an “Epic” story for RC1 critical fixes to correct some further routing bugs. These fixes are being implemented in next week or so.

Another minor version roll out is imminent (0.4.7)

RC1 is starting to be packaged and continuous work around this each and every day.

It was most exciting to hear comments about horizon planning for both Substratum Host and Cryptopay!


The actual Show and Tell segment takes up the bulk of each weekly meeting. Team Members take turns to ‘Tell’ the team and community of code commits or fixes, and also ‘Show’ on-screen changes or new implemented features to demonstrate their use.

Outlined below are the major points to digest from this week!


Creating a Sub Config File — Dan Wiebe:

Dev team implemented a ‘config file’ into Node software, because at the moment, the CLI (command line) startup command has a lot of parameters that need to be typed each time a user wants to run Substratum Node. (or integrate these parameters or settings into the environment)

Issue is, there could be prying eyes viewing a user’s command-line if typed each time. Not to mention it is cumbersome for testing and for users to simply start node on machines where their settings are not changing much.

Accountant scans & add delinquencies to ban — Dan Wiebe:

The Accountant module within SubNode keeps track of ‘who owes what, to who’ for routing, and from time to time refers to a receivables table of these values to ‘balance the books.’

Bills accumulate towards a threshold (based on a curve) where the bill is to be paid for consuming.

There is another threshold noted as a delinquency limit, and when these limits are exceeded, the accountant will add that consuming wallet to a ‘ban-list’ in memory, because their debt for consuming hasn’t been paid.

Based on this, a user using a wallet on the ban-list will not have their packets routed — meaning that their node will not be able to consume, unless they pay their dues, or consume across Node with another wallet with a positive balance.

Another upcoming change on this item will be that nodes with a consuming wallet on the ban-list will be removed from the ban-list when they have paid enough of what they owe.

Node Gossip not properly introducing 6th Node — Dan Wiebe:

Currently the node neighborhood is designed to have up to 5 immediate neighbors connected to each individual node.

A bug was detected through testing with Brian, where a 6th node was not being properly passed on to another nearby neighbor to attach to, when it tried to connect to a node that was ‘full’ (already had 5 connections). There was a discrepancy with passing on the newcomer node to another one nearby. The gossip protocol was not checking with the neighbors being passed to about whether the newcomer was already a connected neighbor to them. Due to this, new nodes joining were occasionally being passed back and forth and would not connect properly to neighbors nearby.

No more bootstrap nodes — Matt:

The bootstrap function has been deprecated and the code cleaned up to remove this. There were a few parameters and command references now removed.

Now nodes can connect freely with neighbors, by debut and passing gossip.

Removal of EntryDNS crashpoints — Dan:

During testing, there were Node crashes when an EntryDNS query failed trying to receive an invalid DNS request from a browser process. There was a fix put in so the Node doesn’t simply crash when this behavior occurs, it just gives an internal error to the EntryDNS module.

SHOW — Demo from BJ:

B.J. put the updated software GUI on-screen to show the changes. The updated SubNode GUI shows a much more streamlined interface. The forms only show the required fields to Serve and Consume, and there will be a ‘Config’ option in the Node menu to allow configuration of settings into the config file that was integrated.

Changing the Node status (between Off>Serving>Consuming etc) will retain the configuration that was inputted by the user.

Note — configuration settings only persist while the Node instance is running. It will be embedded in future versions.

Also in future version coming soon, there will be data shown in the UI like statistics of routing, earnings etc

Account Payment Detection — Matt:

This module scans the blockchain for the highest block number, and detect recent transactions to update the accountant module and payment statuses (keeping the books up to date!). Block number stored in database, so scanning can be done in forward fashion.



Over last 9 weeks, the team has seen constant progression, with each weekly iteration drawing a week closer on the countdown.

The team ‘just scraped through’ this week on the countdown, with the tech debt involved this week working on the 3 defect cards (it’s Brian’s fault lol).

“Team worked really hard, and getting stuff pushed through.” — BJ

This is a moving target — scope can change, people could be off sick etc!



“This week reflects the work we did with the testing community… the 29% and defect fixes…”

BJ explained that the team is now ‘playing whack-a-mole’ on remaining issues we don’t want in version 1.

57% capacity was on the ‘planned work’ side so that looks really good.



Guest Questions/Comments during meeting:

From Brian — the team ‘kicked butt’ with the bugs! Things looking good with 5-node and 6-node clusters in current tests! Routing works perfectly with HTTP & HTTPS!

Will continue to test in isolated test environment.

Had to delete the sqlite database (node-data.db) when restarting nodes and building new networks.

Steve Swing — Name and location of database (.db) file has stabilized so it may be prevalent, but is easy to work around now with that.


Overall seems like another weekly ‘notch on the belt’ for the project timelines, with a successful work-week accomplishing fixes and cards leading to v1 RC1 (Release Candidate 1). The sense of relief in the discussion was noticeable from the team, since it was a short work week in the USA, but the meeting was also quieter than usual as some regular attendees were not present.

Stay Tuned for the next update!

Disclaimer: I am not an official Substratum team member, just a community supporter. This write-up is interpretation of the Show & Tell meeting and data released from the team. None of this is to be taken or construed as investment-related or as financial advice.