Dev meeting in 11 hours.

are tests on master working for other people? I'm seeing failures in functional and unit tests

Since no actual details seem to be forthcoming... unit tests are known to fail in DNS tests in some setups.

woodser: they seem to be working

Snipa, binaryfate, moneromooo, selsta, xiphon meeting in an hour

test failures with dnssec were pretty common for me

seemed to depend a lot on specific version of libunbound you built

but also depended on libunbound correctly detecting cipher support in OpenSSL

sarang: also

meeting in five

roger

o/

Good morning/evening everyone. It's time for a dev meeting.

hi

We've moved these to be really on an as-needed basis, and it was requested by binaryFate to have one. So let's get started.

1. Greetings

hello!

Hi

The meeting issue ^

Hello

Hi

hi

Here as well

2. Brief review of what's been completed since the previous meeting

hey you sexy community you

Let's keep this section just to information updates. Discussion on research or dev points will come next.

should I start?

go for it vtnerd

Hi everyone - my first dev meeting attendance :-)

oh whats the difference between information updates and dev points?

Meta?

oh whatever, I'll start and someone can stop me if necessary

vtnerd: meaning this is just saying what you did. We can discuss if people have issues or questions a bit later.

I posted a github issue explaining my attempts to improve the zmq-serialization in particular for perf and multiple formats. it could also be applied to epee, which would accelerate p2p traffic (syncing, etc)

it may have been too much macro and c++ for some, but that will have to be left to others

I've also been hoarding code that gets rapidjson to write directly into a `byte_slice` to remove the last remaining copying on zmq-json output

I've been hoarding it waiting for reviews to settle, but I'll probably just post it shortly even though it blocks with some thing else

this can also be used to replace std::stringstream in epee binary output and with some more work the json output there too (definitely faster, don't have numbers off-hand)

lastly, zmq-pub is finished and only needs unit tests. txpool and block events are supported

underscore not hyphen, sorry. dont have much else that i can think of. might be some more minor perf stuff to serialization

fin

man github is weird, github.com/monero-project/monero/co…r...vtnerd:feature/zmq_pub?expand=1 might be more helpful for zmq interested people

nice. Thanks vtnerd. Does anyone else have an update?

Ok, let's move on to 3. Research items for discussion

sarang: is there anything you'd like to see is/was/has been in the pipeline and want feedback on?

More progress on CLSAG, which I see has its own agenda item

yeah, 5 and 6 will probably be merged.

Triptych and Triptych-2 have their preprints out

Multisig and join-type math is being worked on for those

There has been talk of Janus mitigations that would add a little bit to transaction sizes; I support this idea, but it didn't seem to gain much traction when brought up a while back

One proposal for Janus would also provide better uniformity based on subaddress destinations

i.e. each output has its own tx pubkey, and then there is a "global" per-transaction pubkey for Janus testing

How much larger would txs be?

Most transactions would see 32-64 byte increase

Keep in mind that CLSAG would already decrease typical transaction size by something like 600 bytes

and the benefit is better uniformity, ye?

and detection of Janus attempts

Note this does not imply extra verification time for the whole network (aside from parsing out these values)

I think the subaddress uniformity benefits are underrated personally

Only recipients perform the Janus computations

Would that mean subaccounts are as secure as separate wallets?

and the Janus computation is pretty trivial

From a Janus perspective, yes

Recipients could detect attempts to Janus-link addresses

at which point they would...?

Wallets could either choose to accept/spend the funds, or treat the transaction as "not destined for me"

(the latter is safer)

"There has been talk of Janus mitigations" <-- best place to find these discussions?

Has been brought up during many -lab meetings and other discussions, as well as research-lab repo issues

Can discuss at this week's research meeting too, if desired

Maybe a small write up of this would be warranted? Pros, cons, background info, etc.

Would be very interesting. Not saying you to do it sarang. Get someone else to learn and writ eit.

*write it

See: monero-project/research-lab #71 and monero-project/research-lab #62

Ok, anything else on this topic? If not we'll move forward.

What would be helpful for me is to know whether or not people support such a size increase for this benefit

If not, we can drop it and provide better user education on subaddress limitations

If so, we can hopefully move forward and deploy the mitigation

Never offload to users what can be done by the software.

When first brought up, there was a lack of general dev support

I would probably take that for 64 bytes anytime ...

I say yes deploy, just because of UX 101.

I think it's a good idea for subaddress protection and for uniformity

Note that it doesn't have to be consensus, but could be

The wallet would probably provide a big warning to users, but funds shouldn't be unspendable. If someone actually pulls off a Janus attack even with mitigation in place, that would be very interesting

I also think it's needed to make subaddresses function as people believe they do

UkoeHB_: an attacker could easily post such a txn

More likely is other wallets failing to implement it correctly

it's up to the wallet to decide how to handle it

At the risk of stating a potentially unpopular thought, I think making it consensus would be necessary, since subaddresses are a part of our ecosystem and are here to stay.

So to recap, under the assumption of CLSAG, txs still become much smaller than they are presently

This makes it fairly idiot proof, no?

and still become faster to verify

Consensus is questionable since it's not verifiable

UkoeHB_: I mean consensus in terms of the presence of the expected number of tx pubkeys

All you can require is a few extra bytes exist in the tx

yes

Anyway, please carefully consider this idea

Well, unless there is other thoughts on this, we can discuss at the next MRL meeting.

Let's move on.

I think it would be ok to require that, since ECDH info is already a precedent

I'm going to merge items 4, 5, and 6

Dev items for discussion, D++ & CLSAG, and v0.15.1.0 discussion respectively

Let's actually start on the tail end of that. v0.15.1.0. When? How? Why?

vtnerd: is that supercop stuff ready to merge after review?

yes

it can be used within monero or another project, or be built and installed as a standalone lib

Is the wallet integration also PRed already?

or is that not required

yes, but its marked with do not merge because it requires the supercop merging first, and then a pin to the hash of the supercop

ok, I would like to get Dandelion and supercop in for v0.15.1.0

do you think that’s realistic?

you know, that was probably never mentioned in a -dev meeting either so yeah more stuff to mention

um, the supercop might be tough for reviewers, that will be the biggest holdup

can we get either context or a link for "that supercop stuff"?

see the single PR

with very little additional efforts the meeting would be easier to follow for bypasser/newcomer

thanks

I’m on mobile so I can’t post links :)

it accelerates wallet scanning on x86-64 boxes, using code from Daniel Bernstein, et al's supercop repo

Waiting for quite some time, right?

the ed25519 code could not be used directly, modifications to work with monero specific stuff was required, thus the tough review (its hacking a C crypto implementation)

yes, although expected giving the situation

who has the skills to review ASM?

or do we have to review the ASM?

Has anyone outside monero dev expressed interest or been recruited to look at it?

the ASM is very similar to the existing supercop code. you can diff it between the existing "base" code and my new additions

ok, that’s probably reviewable

but you would still need to know a bit of ASM to understand what its doing and why

yes, if you trust the supercop amd64 ASM, then its easier than reviewing completely from scratch.

moo and I have a discussion about this in the PR for the files that need to be compared

No (friendly) fork with enough traffic to have interest in this as well?

To help to review, I mean

I think mooo was interested in reviewing it but I don’t want to speak for him.

Isn't the point of C/C++ to avoid ASM?

we want ASM crypto to speed wallet scan up

why not write everything in ASM?

crypto libs man

because its tough to get the perf + constant time code

I think it's a well-defined and reasonable use case for ASM

i don't buy it

did you reach out to Bernstein and co-authors for a review?

that ASM I wrote keeps the code constant time, whereas some other forks _definitely_ dropped that feature when taking the supercop code

it's never a bottle net to begin with

neck

fuwa, writing constant-time code in C is harder than you think, please read

I saw the RP, I can't read it :p

So it would not only be a little faster, but also better concerning timing attacks?

if you drop the constant time behavior, crypto code will be much, much faster

it's not slow, even on my phone

sarang could probably comment on this with regards to the variable ECDH thats lurking in some sports

yes, it is

look at the perf numbers

There are plenty of spots where we use vartime scalarmult operations in particular

this is an optional wallet feature that I can change to disabled by default

- but wallet viewkey scanning ?

mostly on the verify side, no?

mostly, yes

on the topic of supercop, have you used ctgrind to evaluate this yet? github.com/agl/ctgrind

no idea this existed, but awesome

I'd forgot about that PR since it's elsewhere. I'll try to go through it soon.

So are we looking at maybe a couple of months if we want to get both of selsta's requests in this next release?

I wanted to get v0.15.1.0 out in 1 month or so.

Maybe it would be better to only include Dandelion and then supercop + CLSAG in the next HF

Let's discuss CLSAG real quick

sarang: can you comment on how close it is to production ready on both a code and audit level?

as for perf numbers, scanning the whole blockchain went from ~46 mins to ~20 mins on a 2014 mac mini. An older piece of hardware, but not _that_ old

did anyone reach out to Bernstein and/or co-authors for a review?

You mean hiring Bernstein for a review?

I mean just asking for a friendly quick check from one of the authors or maybe a student of theirs but anything is possible.

D. Bernstein was speaking at the Monero assembly at the CCC a year ago, on a panel on privacy and stuff. Who knows maybe there is enough interest.

Nice. Maybe someone who can write well can reach out.

Just letting them know we have this thing going on and are looking for reviews, see where it goes

vtnerd: did you see moneromooos faster fetching of txs PR?

That's daemon only, while vtnerd's wallet side. They won't interfere.

But should result in even greater speedups?

together?

Yes, if you run both on the same machine.

ok :)

Well, less load. But it's likely one of them will be the bottleneck, so the other will wait.

The wallet's probably the bottleneck there.

So the daemon will be done faster with the request, but will sleep for longer till next request.

rehrar: maybe you can add writing Bernstein on a todo so that we don’t forget it

Already done.

Ok. Ready to discuss CLSAG?

Sure

I had been waiting for some time on changes/approval from coauthor suraeNoether

However, he just recently gave me permission to make the changes and post the revision without his further input

So, I'm completing that, which should be done within a couple of days

Optimized CLSAG code has been merged into moneromooo's now-rebased `clsag` branch

My local copy of that branch has some new plumbing to help with trezor/ledger support

I'm now working to coordinate that, so trezor/ledger will be (ideally) ready to go right away at upgrade time

The 600 bytes reduction is already visible on that branch or assuming further changes?

The 600-byte (for a 2-2 txn) has not been changed throughout CLSAG's development

So yes, a transaction generated from that branch will see the expected size reduction compared to MLSAG

and will see verification speedups

some of which are relatively new

this is awesome :)

Because of some speedups to MLSAG that are already in place, the current speedup going MLSAG->CLSAG is around 16% IIRC

I can't comment on how trezor/ledger development will take, but the preprint will be on IACR within days

and my device-specific plumbing changes are minimal and could be merged to the `clsag` branch whenever convenient

and it's best not to put CLSAG in the release until Ledger/Trezor is properly integrated, yes? Because then that puts us with very angry users.

yes

I agree

However, I'm actively in contact with those folks

They are using that `clsag` branch for their work

So if selsta wants to try to get 0.15.1.0 out next month (And so far nobody else has given a timeline) then maybe it's too soon for CLSAG indeed? When would we do a hard fork release after that? Two months from now? Three? When ready?

Is there a need to rush it?

The upgrade, I mean

CLSAG would not be included in v0.15.1.0

CLSAG would be v0.16.0.0 due to hardfork.

ah, I see.

my bad. Sorry.

Review on moneromooo's `clsag` branch would be welcome

and/or the additional pending commit on my `clsag-device` fork of that branch (which adds device-specific plumbing)

Would May or June be when we can start looking at v0.16? Just as an idea of a date, not committing to anything obv.

I think we can think about a date once we know how long the audit is going to take.

If Teserakt are chosen for the review, they had previously indicated a very short turnaround time

that's up the audit workgroup

can people who have participated in the audit workgroup be named so we can get this on their agenda?

Reviewers would need the preprint (just a few days away) and the finished code

AFAIK sgp_ has been coordinating that effort

ok, thanks.

anything else on this topic?

I'm offering my support as a researcher, but will not be making any decisions on that

(I think it's good practice to have that handled separately)

Maybe at some point we need to give a timeline or soft deadline to ledger/trezor? With some good margin on top anyway before the freezing for hard fork. To have some visibility and not depend on external things a bit opaque.

binaryFate: I only reached out to them a few days ago, so they are reviewing the changes they expect

They've indicated the changes should be straightforward and relatively minimal, FWIW

sounds good

I'll follow up if/when I hear more details

I think code reviews are extra appreciated in the next days / weeks so that we can get a lot in for v0.15.1.0. We will fork from master branch. We will also need a bit of time for testing before release.

Were there any outstanding CI issues with current master?

no

ok

compilation was failing yesterday but it is fixed now

Forking after going through some PR / merging backlog?

yes

Nice

Ok. We're running a bit long, so I'm going to move to the last item.

These days i'm thinking of the possibility of organizing a hackaton for 0.15.1 and let people fix bugs. Maybe we could open a CCS for a prize

binaryFate: you wanted "seeds" added to the agenda. Can you open us up on this?

It's simple: the situation with the seed nodes is sub-optimal. We're right now at 3 up, because we raised the issue last days, we only had 1 up before for quite some time.

mooo PRed a node.

What, on mainnet?

Same situation happened 2 months ago, we raised the issue and got 2 nodes back up (thanks to fluffypony), but we eventually went back to 1 node again

Yes mainnet. So we are 1 single node from newcommer downloading Monero and being unable to sync without going on forums to find extra commands and stuff

So, was ok but close call

You can see seed nodes here: community.xmr.to/xmr-seed-nodes

I knew that there were no seeds at all running for testnet, but mainnet ..

Actually testnet and stagenet had 3 nodes up when mainnet had only one...

Cool

I’m interested in traffic requirements for a seed node.

Out of the 3 nodes up, I believe one is ran by Snipa and 2 by fluffypony. The other afaik we don't know

Also there are domains in the code, to be used to get seed nodes, but none of them have any dns records

So there has been some kind of ... bit rot :)

Yeah been slowly driffting downward. So how do we improve at next release?

I'd suggest in general to get 3-5 domains up, and maybe a dozen seeds up from trusted volunteers.

Remove any existing seed where we don't even know who is supposedly taking care of the node

I would be quite interested in running a seed node, for example...

Also, someone in #3425 was interested in running a seed node. That was a while back though.

I can vouch for yugyug that I know IRL (we work together).

We'd need to find these volunteers. Do we just put out a PSA and choose the most trusted responders?

In might be useful to have a ball park for traffic requirements of a seed node.

fluffypony: do you know ? Combo of print_net_stats and status.

Well, you can restrict at any time, probably most important to get to know peers for other nodes, no?

yes, if a $10 VPS (~2TB traffic) can run one it will be easy to find volunteers

selsta: for that price will it have enough storage?

pruned node, yes

indeed all the network wants from a seed is that you can serve a quality up-to-date peer list, not necessarily syncing much from it directly

Exactly - assuming the main role of a seed-node is intodroducing new peers it should be enough

if $6 you can get 100GB SSD storage and 20TB traffic.

for*

at hetzner*

In that case then yes, shouldn't be much an issue

and i confirmed they dont care about running i2p nodes or xmr nodes.

kinghat[m]: Hetzner is nice but probably the most provider, we should diversify a bit

most used*

do they accept btc or Monero?

Would surprise me

i dont think so

eh

ok. binaryFate anything else you wanted to say or ask regarding this?

everyone who wants to run a node should open a PR adding their node, mooo did this recently

next steps about seeds? Visible PSA or just mentions here and maybe -community?

run a seed node*

ok. selsta shall we wait to merge them? In case there are 50, we will want to select

selsta: this may be the way to handle it. Which ones get merged can be discussed on the PR in regards to tr;ust.

yep

maybe mark them as WIP for now

discussion can take place in the PR if the user is new

rehrar: I would like to add a quick meeting item: moving back to Github for all repos

ok, discuss

selsta: you have the floor

anyone against this?

I think we have tried gitlab for a while now and there are some restrictions / limitations.

What about the CCS stuff?

What is that, in addition to the website?

Doesn't that require some GitLab-specific stuff for the backend?

It would be best to leave it online as a repo backup but move back to Github.

sarang: afaik xiphon coded github code for it

simply pulling the css backend from github should be enough no?

Community discussion happens as GitLab MR comments

Which presumably would be lost

sarang: Xiphon and I did some work for Zcoin to launch their CCS. They use Github, and Xiphon made the stuff for it to happen.

So, in theory, switching over shouldn't be too big of an issue.

rehrar: is there an advantage to moving CCS to github?

is it possible to seamlessly switch over to gitlab operating as a mirror if github gets ugly?

I assume that CI stuff is not important for CCS repos

it would IMO be best to have everything in one place

are the current limitations listed/discussed somewhere?

sure, but it must be annoying to run a whole gitlab instance for just CCS things.

the existing discussions would stay online

There was a github issue comment

rehrar: wouldn't there still be a gitlab instance for mirrors?

(last 2 comments)

yes, gitlab instance would stay online as a mirror

binaryfate: i tried to make a summary: monero-project/meta #236#issuecomment-605023567

IIRC mirroring only works the other way.

It shouldn’t be too difficult to write a script to mirrors the repos.

(unless manual scripts to pull I guess)

It would be nice to have things like easier 2FA and SSH access for everything

(that's a huge gitlab limitation IMO)

and working CI for PRs

ssh is not gitlab's fault, it's cloudflare's fault.

the only thing we should look into is a regular backup of all Github discussions

Well, but same end result for users ...

I don’t know if Github has an option for this or if we have to use a scraper.

regular backup of all Github discussions <- yes, we should look into ways to preserve history in case things gets bad

To mirror manually ? A set of git pull.

isses / pr discussions

issue*

moneromooo: true regarding SSH, but we still don't have it :/

Oh, sorry.

GitHub-only would also remove some of the common account issues people seem to have

regarding forks, and github credential use, etc.

we can check github.com/marketplace/backhub

binaryFate: maybe they offer a free service for open source software

they use github API and seem to be able to get metadata

else I guess $12 or so sounds okay.

There has to be since gitlab could pull all the history.

what about microsoft thing? Was primary driver to move...

The issue comments discuss the problems with gitlab

regarding accounts, CI, etc.

We still don't have anybody speaking out against the "next spawn of satan" :)

(see issue title)

FWIW it seems that MS has done a good job of not interfering to the extent possible

I personally don’t think that Github has gotten worse since the MS acquisition.

^

They always do until they fuck you over.

sarang: it's still a time bomb. The point is to have a self hosted platform, but gitlab that supposed to be the best, turned out to be not that great

fair

If there is an issue it the future we can migrate back.

But FWIW, seems that having self-hosted will always result in fewer developers being on that platform compared to something like GitHub

That's not really a GitLab issue, but a "where are people" issue

Anyway, nobody is at the moment seriously voting from moving *away* from GitHub for the main repos, right?

sarang: i thin the problem is that all main repo are on github excpet for one, the website

For better or worse, people are on GitHub, which has good CI

And using monero rather than bitcoin, or Linux rather than Windows. We still do :)

Well, I guess some here use windows and Bitcoin. But still.

i see the problem of the missing people on gitlab more like a problem caused by the split than something related to the platform itself

I mean people still have to signup for the wesite repo compared to Github so there will always be less people IMO

Is anyone here against migrating back to Github for all repos?

And keeping the existing instance for its discussions etc.?

I'm indifferent.

(useful esp. for CCS)

I kinda agree that the CCS is probably best left on gitlab in theory. Whether it's worth the expense though, I dunno.

IMO visibility and ability to attract contributors should be priority especially in these turbulent times (cons: many projects will probably die, pro: many people are stuck at home with some time), we can adapt to new things if there is a spawn of satan situation

has anyone ever contributed to monero by stumbling across it while surfing github?

I kinda like that people have to jump through an extra hoop for the CCS though.

I think I might have. Not sure.

It's a step 0 for people requesting money. Can't even do that, we don't want their proposals.

I don’t care about CCS repo. Mostly site repo.

i think would be better to move everything, but i see why could make sense to keep CSS stuff on gitlab

let's start with site then

CCS is not urgent to move, but can be down the road

sounds good

+1

ok. Anything else in this hour and forty five minute meeting?

o_0

cool deal. Let's break.

Did we discuss CLSAG audit(s) already or?

Thanks everyone! Bai!

The update TXT records. Would be nice to get them updated.

fluffypony ^

I pinged pony, but he might see that in like a month.

After the pandemia, I suppose

Ah yeah. Packets should stay on the local network.

fwiw we're working on me having dns access as well, but the transfer isn't done yet. So next time can be faster turn around.

Actually... Given pony lives on planes, I wonder if he's stuck in a plane doing figures of 8 above somewhere for weeks...

<dEBRUYNE> Did we discuss CLSAG audit(s) already or? <= Found it in the logs, nvm

Like I said, get a monerojet, and I shall get type rated for it :)

Still? Even after scaling back Monero involvement?

DNS is by nature centralized

Let's CCS a monerojet, sounds like a priority

fuwa-prpr we'll have 2 persons with access instead of 1, that's something

for backups there is also: hackage.haskell.org/package/github-backup

I'm off, thanks everyone, be safe

So what would a `monero-site` migration to github look like?

In terms of MRs, comments, etc.?

i think issues would be preserved, but i don't think that will be the cse for MRs

for the comments we just leave the repo up on gitlab for reference and as a backup i would say

yep

Man, it would be nice if Git maintained all that stuff as part of the repo

So migrations could be trivial :/

not the case sadly

truly

Timeline estimate for this migration?

i guess that depends by core

luigi1111 ^

luigi has to reenable issues and edit the description, then we can migrate?

I can do whatever as long as it's not too time consuming

no, should take 1 minute

Let me know when you do, as I'll have to disable the auto mirror on gitlab.

moneromooo: I’ve read that Gitlab mirror works in both directions. Can you confirm that?

it seems like the mirror stopped working anyway, last update is 20th February on Github.

Only if you pay, AFAIK.

hmm. as always :D

selsta moneromooo : should've added to the discussion above. my machine with ryzen 3 + samsung pro nvme seemed to be bottlenecked by fetch (daemon) speed, whereas any of the slightly older machines bottlenecked by cpu (crypto)

but thats a slight guess

nearly all machine should benefit from the daemon fetch code and all x86-64 machine benefit from the crypto, with some overlap

what was the reason that we bundle our own unbound?

probably to have a known version and a known set of bugs

okay, they keep putting out new versions with more bug fixes

so we should probably update to latest release, but then again this might introduce new bugs

I think mooo has a PR open to update the submodule

Oh yes, I'd totally forgot about that one. It needs someone with repo write access to push -f IIRC.

moneromooo: should we set it to master or v1.10.0

maybe you can update your PR again as a new version is released in the meantime

I don’t really understand how this submodule thing works :)

I guess latest stable release would be best.