Substratum Dev Team Show & Tell — 5-July-19 (i35)

Show & Tell weekly sessions are an excellent insight into the challenges and triumphs facing the Substratum development team. For those following these Show & Tell discussions, the format of the progress breakdown has just gotten slightly more tense and exciting, because the team is down to the last handful of critical items (cards) to move to the first Release Candidate (RC1) of the SubstratumNode

As always, B.J. Allmon heads the meeting by outlining the Substratum Node Version 1 Strategic Goals:

  • Priority 1. Make the Internet Free (as in freedom) and Fair. — “As a SubstratumNode user I can allocate my spare computing resources so that the Internet can be a free and fair place for the entire world.”
  • Priority 2. Earn Cryptocurrency. Change the world. — “As a SubstratumNode user I earn $SUB as content is routed through my node so that I am incentivized to make the Internet free and fair.”

BJ gave a bit more clarity here about how the Node vision is being developed in real usages:

In Priority 2, ‘content’ is a big word — it could mean a lot of things. It could be routing services, exit services or even blockchain services eventually. This could be where people don’t have access to a GETH miner or a blockchain service URL (like Infura), they can get the block information from users on the Substratum network.

Many of the Tactical Focus Items listed have been done for a while. Most of these have been done for the purpose of RC1 and the others that are Work-In-Progress will be built-out further after RC1 to provide more in-depth features.

GUI updates have notably been challenging and involve a lot of work.

The ‘Show and Tell’ segment

Still plugging way, and we are looking at the data differently from here on out, with the remaining cards. — B.J. Allmon

The team review the completed work for the week, including Telling the story of each card, or Showing the completed feature.

Setup DNS Utility to require rustfmt

This was actually completed for all the sub-projects within the repository, as well as the DNS Utility. Basically just reduces noise in the teamPersonal Repos.

There is a known bug in the DNS Utility that needs to be fixed for Windows, so this will assist in cleaning that up (tech debt)

Prevent fraudulent consuming wallet usage — Steve Swing

When your node builds a route, it will take the network public keys of the route nodes, and provide the proof of having the private key for the consuming wallet being used to pay for the services — this produces a signature. This lets all the nodes on the route know that the requesting node has:

  1. Provided a wallet for payment
  2. Can sign transactions for that wallet

This will prevent node users consuming on the network without a valid wallet to pay for services.

This does create an incompatible version with users who have downloaded the v0.4.8 binaries, since this updated code is not in that downloaded version.

Another main function of this completed card, is that the smart contract of the token is also called into the signature, so testnet won’t be compatible with mainnet.

Implement clippy linting for dns_utility:

This was a tech debt item

Clippy provides scans of the code, and provide hints that you are running certain code functions.

SHOW — Wallet Password Unlock for the GUI — Steve Swing:

BJ put the SubNode GUI on the screen.

To understand the wallet functionality a bit more, its best to clarify how wallets and wallet passphrases are derived (not just specific to Node wallet generation)

Passphrases and passwords are two separate things.

The mnemonic passphrase is the additional word you include with the BIP39 mnemonic words, that creates a more secure wallet. (usually it’s a 13th word or 25th word when generating new wallet seed words)

The words are used to generate a seed value.

The seed value can be combined with a wallet derivation path (n/44’/60’/0/0 etc) and this produces a key pair (wallet public address and private key)

To keep that seed secure, a password can also be collected to encrypt it.

The mnemonic seed value is stored encrypted in the database, but the password is used to decrypt it and provide the wallet info (mnemonic seed) to the Node itself.

Essentially all of this is in place so Node can pay bills for routing!

The GUI on screen showed the prompt that the user sees to provide the Password to unlock the wallet being used for consuming.

BJ explained that the format of tracking progress has changed.

From this point on, velocity averages will not help predict the final time frame to complete the RC1. BJ said he will tweet updates as the remaining cards for RC1 are completed.

2 cards are in flight and are really close — they unlock two cards in the To-Do bucket. One of the four remaining is also dependent on another being completed first.

Therefore RC1 completion cannot be predicted based on how fast the team previously were completing cards, since they are down to the final few and they are codependent on each-other.

Thankfully the work on the routing for RC1 has already been completed.

Dan Wiebe explained more on how the cards became more technical resulting in several of the final ones remaining:

We assumed that every time you started Node, you wanted to use the same earning and consuming wallets and could store and reference them in the database. However, it was discovered that changing the consuming wallet can cause issues with tables stored in the database.

So with this challenge in mind, the work relating to the consuming wallet was split into several cards.

BJ clarified there is still the ability to have several nodes running that are sharing the same earning wallet, but there is some complexity to work around, as the accountant module will be tracking all the transactions relating to the same earning wallet across many nodes.

It’s definitely something that needs to be thoroughly tested in the testnets.

Team Capacity Utilization overall initiative looks great!

This week the team had 2 cards that were critical RC1 cards, and two others that were tech debt that relate to Version 1, but not necessarily RC1 — that was played to keep several of the development team busy working on constructive items while others play out the RC1 cards.

For those who want more technical insight, I tried to capture some of the Guest Open Space chatter. If you don’t want to ‘geek out’, you can stop reading here!

Guest Questions/Comments during meeting (paraphrased!)

Aussie_Crypto — Question from the community was do you have plans to have testers try to attack Node on a clandestine and monetisation level

BJ & Steve — Yes definitely. What’s challenging is that many of the High-Level Security Execs we deal with are not experienced in hacking in this area.

Andreas — Offering a bug bounty would be a great way for this — community could be incentivized for this.

Brian — Since the latest commit has breaking changes, the community isn’t able to continue testing. So do we anticipate a v0.4.9 version so the community can test a pre-cut binary so the testing velocity can be continued?

BJ — its a good question — we will have to see how we sit mid-week

Steve — it would be good to see another minor release, as the next few cards will have compatibility changes

Disclaimer: I am not an official Substratum team member — I am a community moderator. This write-up is interpretation of the Show & Tell meeting and data released from the team. None of this is to be taken or construed as investment-related or as financial advice.

Substratum Dev Team Show & Tell — 28-June-19 (i34)

Show & Tell weekly sessions are an excellent insight into the challenges and triumphs facing the Substratum development team. Each week a set of slides is released and a group conference is held with dialogue and examples of work planned, completed and lurking on the horizon!

This week has a small number of items on the Show & Tell, but they are very important.

As always, B.J. Allmon heads the meeting by outlining the Strategic Goals of v1.0.0 Substratum Node:

Priority 1 — Make the Internet Free (as in Freedom) and Fair

Priority 2 — Earn Cryptocurrency. Change the World

Tactical Focus Items are outlined listed and briefly mentioned as Work-In-Progress

Many of the listed items are completed for Release Candidate 1, and this is very promising in terms of having a secure and stable candidate to fully test, leading to the final Version 1 (v1.0.0) release!

BJ explained that the team also has spent a little time mapping out CrytoPay flows, and also integration ideas in the Amplify product suite and Amplify wallet.

The ‘Show and Tell’ segment this week shows only a small number of cards, since there are very few cards left to work on for RC1.

But the things in it are GOOD! — B. J. Allmon

Outlined below are the items to report this week:

TELL — Proxy Client timeout charges for 0-byte — Dan Wiebe

Actually fixed now!

Dan spoke a little about this last week. The issue arises when a connection or stream is broken between a browser and a server. If the browser closes or stops requesting the connection to a website for example, SubNode was not communicating that to the website server (including if this happens from the server end also). This arises because SubNode sits in between the browser and the server in the network layer. So there needed to be a way to send the “Drop-Report” across the network to the exit node to inform the Server to terminate the connection. And vice-versa for the server to report to the originating node browser if that connection drops or ceases to connect.

Tell — Write logs with closures and macros, not functions — Dan Wiebe

When SubNode writes logs, there is usually a string written which is input into a log function. The level of the logging determines whether the logging subsystem outputs the log text or not.

In many cases the work involved with the logging was not required, and performance and resources were being wasted doing this.

So instead, the function and string work will be avoided if the logging is not to be written when your logging level is set very high (to WARN or ERROR)

So this actually boosts performance when you don’t have the logging level set to DEBUG or TRACE

SHOW — Release Notes for v0.4.8 — B.J. Allmon

B.J. Allmon went over the release notes and fixes that were implemented in the recent minor release.

The main reason this minor release was pushed out, was so the fixes could apply to testers using the GUI, so they can also have the benefit of the performance fix with the accountant thread.

SHOW — Make logs consistent — Dan Wiebe

Logging was pulled up on screen, and the old versus the new were compared in text form.

Overall, there is now less logging being written that was wasting space and creating extra parsing. This includes referencing the libraries being used that get output into the logs.

So the logs are more streamlined but also consistent over the different log level outputs.

Incredible to see the 1-week figure on the trajectory for RC1

Incredible to see the 1-week figure on the trajectory!

But it can be misleading even though things are looking good.

First time in the history of my software development career, I’ve seen this same 2-week forecast over a 2 month range! — B.J. Allmon

B.J. Allmon explained that 4th July is a USA holiday where they aren’t working and the talk in the community has been it would be great for excitement and marketing etc.

But the team is unemotional about it, so they are just trying to get the work done.

In the To-Do bucket, there are just 3 cards, but only 2 can be worked on in parallel — the third are dependant on the other ones, so it’s an order-dependant process to complete them all.

Team Capacity Utilization

B.J Allmon added an ‘Ideal’ benchmark bar, and the ‘Initiative’ bar, which shows across the majority of the work done on the SubNode initiative.

The recent bar showing ‘Other’ related to the work on the logging, which wasn’t critical to RC1, but allowed members of the team to be productive on other enhancement.

The team has four things in flight which are really close.

Stay tuned for another Show & Tell next week with the Substratum Developers — hopefully its an announcement of the Release Candidate 1 being ready for testing in the real world!

For those who want more technical insight, I tried to capture the Guest Open Space chatter. If you don’t want to ‘geek out’, you can stop reading here!

Guest Questions/Comments during meeting (paraphrased!)

Andreas reminded of the changing (dynamic) IP concern

BJ — Some ideas are being discussed, which will be exciting to see how it can be implemented. The team agrees it is totally doable.

Steve — we need to investigate what happens to the IP on the ISP end, and how it affects the nodes connected. It would require another neighbor on the network to ‘notice’ that the IP address changes, and then be able to reintroduce the node into the network again.

Cuba — the feedback in the community is that the performance has improved!

BJ — has seen a lot of the State error coming up in the test community and lots of items noticed with Windows 10. The team is getting to that, and a lot of work for Windows users is in the backlog and will be worked on for that.

BJ — we have considered the test community suggestion to have some official nodes running to start the network. But we wouldn’t want to ship the software with nodes already embedded in the GUI

Currently the overall UX experience is the ‘Version1 experience.’

We have a lot of ideas — based on things like Proxy, instead of subverting the machine DNS.

The GUI needs to be completely revamped at some point.

We need to rethink that, but now the priority is a functional product that proves that it works.

Brian — the ‘Consume’ button seems to be flawed

Especially in the Windows GUI, as even though the DNS entry is showing up on the IPv4 settings, routing stops working when a user clicks CONSUME button from the SERVING state.

This can be fixed with a work-around by having the State as SERVING, open up the IPv4 properties in your network settings, and set the DNS manually to — works every time so far.

BJ — this seems to be an issue and the fix is in the works

Disclaimer: I am not an official Substratum team member — I am a community moderator. This write-up is interpretation of the Show & Tell meeting and data released from the team. None of this is to be taken or construed as investment-related or as financial advice.