The PR's on hold till someone with domain experience reviews sekreta really.

Though atm it seems to be a direct API for API translation layer rather htan the multiplexer I had been expecting.

i could review it

and give a report

I have understanding of I2P although the monero side of things needs some downloading

not sure about sekreta...I could review but it I think anonimal might be deeper into it than I am

i've been more focused on using available networks together intelligently instead of trying to improve on them

zzz of I2P would be an awesome person to have review it...for the I2P portion

or str4d

neither of them do c++ (at least to the extent demanded)

oh right, they're java guys

and i'm mainly a java guy too

(str4d is also fantastic at rust)

i've been tempted many times to go back to c++ but I think rust might be what I would move into...but not sure...c++ will be here forever i think, not sure about rust

well, we could review it as best as we could and just give a report on our thoughts...maybe those with people who are heavier in c++ could be enough?

Would it hurt to run it on testnet for awhile and verify?

What would be helpful is not so much a C++ review rather than a review of the higher level concepts. Like, does this break any of the protections i2p or tor provide, etc.

"this" being the intended implementation of the multiplexer layer, which is what makes it interesting.

Does it do what it intends to do, in theory.

Does it actually improve on just using, say, tor alone.

yeah it's a bit tougher to review a complete re-write of something versus just embedding something already written...you've got to really understand TOR and I2P to provide a real review

Right. That's what I meant by "someone with domain experience".

And that would be best IMO from someone like zzz and str4d

Or someone from zcash maybe. They could benefit from it (and us from the added traffic too).

I have the Java I2P working and can verify it, but verifying a rewrite... the original developers are best...well, who knows who jrandom is

I think they have at least one network person.

and a lead dev from torproject?

There's a guy from Nym who might be helpful too. He's been trying to rewrite I2P to include tokenization.

He's in the Bisq community sometimes.

After reading the documentation on Sekreta and reflecting back on 1M5, where I see convergence is on the idea of making anonymity networks easier to integrate. Both projects diverge on the reason why though. Sekreta focuses on an assumption that TOR/I2P and other anonymity networks are vulnerable to being 'broken' into and so why trust just one network whereas 1M5 focuses on multiple networks in case one is blocked at

a node, it can use other networks to route around the block.

Sekreta seems pretty low-level in that it is written in C++ of course, but also in that it seems to be focused on improving lessons learned with TOR/I2P, etc. and is trying to create a better experience and more secure multi-anonymity-network as if it was really one.

1M5 is more of a higher layer application router implementing a 'micro-service' event-driven architecture typically found in enterprise application type development.

It obviously comes out of the experience of the author's in integrating multiple technologies to innovate those into a larger web to build something new from a wide array of parts.

It focuses more on leveraging what's already in use in an innovative manner, e.g. using TOR when I2P is blocked to route around the block, and I2P when TOR is blocked...using Bluetooth when a government shutsdown cell towers nearby when activists are demonstrating.

Switching to long-range radio when bluetooth/wifi-direct gets jammed.

Switching to LiFi (Light Fidelity) to get information out of a zone being blanketed with radio jamming.

Their goals are both similar in making it easier to use current networks yet they diverge I believe in how they're likely to be used...who the targe audiences is.

I see them as complementary.

If we can get some devs like zzz/str4d or others the community agrees with to review Sekreta, I will look into what value Sekreta might bring to 1M5 in replacing the TOR/I2P implementation within it. This would delegate the lower-level anonymity network implementations to Sekreta while allowing 1M5 to focus more on integration with Bluetooth, WiFi-Direct, longer range radios (e.g. Locha Mesh & GNU Radio SDRs), and LiFi.

So in the mean time, we could provide multiple methods of proxying to Monero for testing out.

The --txproxy options could be TOR, I2P, Sekreta, 1M5, LOCHA

1M5 would seek to encompass all as that's it's primary motivation....to leverage the best networks to prevent blockage

It seems like Monero is in a holding pattern with

...Sekreta and Kovri until they can be properly reviewed

What's implemented in 1M5 currently is TOR and I2P and now in progress is Bluetooth.

These are the standard TOR and Java I2P networks so nothing new.

nothing to test

The only difference is that it will use an appropriate network based on where the individual is.

It doesn't broadcast too all of its 8 nodes on all available networks.

It selects the appropriate network based on the user's input or the developer's setup if the GUI can not take into consideration the User input initially...GEO IP can also be used to set it up, similar to what I2P does when deciding not to publish IP addresses to its netdb

For example, while in China, it is likely best not to even attempt to use TOR behind any Chinese address

Use I2P by default

And have bluetooth working so that the chinese can help each other share access

This leads me to one thing about Sekreta in that combining multiple networks, it would need the ability to select which networks can be used or not used by the end user based on their situational awareness.

Anyways, it was great talking with anonimal and I'm hoping we can help make monero more censorship-resistance.

I think Venezuela would be an excellent testing ground...would love to see that out ASAP

monerod used 6400MB of ram today.

Had quite a few odd nodes as well connect.

martijn : tewinget added 0mq, but it required a few tweaks to get it working - specifically for the light-wallet-server. the RPC code should be working now

I have a series of patches to improve the performance of writing to json, as 0mq pub/sub is something I'm targeting to get added

_most_ of the patchs are PRed right now, with one other on hold for now. I'm also investigating msgpack performance, and finally found a suitable way to integrate that

but trying to get 0mq pub/sub integrating should come before any msgpack

one of the objectives is to minimize blocking of p2p i/o threads

although monero tx volume isn't substantial :/

vtnerd: Those are interesting developments. pub/sub would be a very useful addition. Do I understand correctly that json rpc is now only implemented on the node side and that the wallet does not yet use it? Is that something you are actively working on or could help with?

the light-wallet-server uses it but not that wallet

I'm not considering changing the wallet; it uses a custom binary protocol that gives it some speed advantages

but it has HTTP overheader, so yeah its a tough one to "clock"

So you're going to skip the http-to-zmq and go right to the pub/sub for the wallet then?

it'll be easier to compre light-wallet-server pulling down txes and the wallet at some point, which won't be identical but close enough to give an indication

I haven't thought about adding pub/sub to the wallet at all

there are downstream people using wallet2 in android for instance, so its going to require some kind of fallback or something

Ah, well then I'm confused. Where exactly do we want to add pub/sub?

light-wallet-server wil be the testing ground since its already using zmq

and there are other people that have asked for it

another big one would be xmrig

you have to poll right now if the pool and node are't on the same box

and even then its triggering a process

I assume that means you lose efficiency due to the miner continuing to mine on the old block before it realizes a new block is there

yeah a bit of time loss, and increased polling uses more cpu

What is the re-org limit on Monero?

What is a re-rg limit ?

I mean the maximum block height upto which fork is allowed?

Hmm. It should work fine till the last checkpoint. Then, it should work fine before, but I don't think that's been tested.

(feel free to, and report a bug if it fails :))

It might require qiute a lot of memory if you reorg hundred of thousands of blocks though, so some nodes might croak.

When monero was hard forked to RandomX, lets say at block height 10000, is there a possibility that someone fork the node and move back to block height 9500? Or when you hard fork , you don't allow re-org beyond hard fork block height.

What does forking a node mean ?

I mean mining more difficult chain, which is not the part of main chain.

So someone starts mining again from 9500 onwards ?

That should work fine, if they prevent reorg to the longer chain.

no, lets say someone mined from block 9000 to 9500, whose chain has more difficult than main chain at height 10,000

If they end up with a longer chain at some point, other nodes will reorg to it.

so in case of monero, reorg happens based on longer chain?

Well, nodes reorg to the longest valid chain they know of.

oh

ok thanks

long here meaning most cumulative work, not height.

yes

I mean the same when I say most cumulative work for someone mined 9000 to 9500 is more than the main chain at 10,000

I've tested reogr past a fork before. I don't think I did with the randomx fork itself.

But I don't see why it would not work. Possibly something to do with the seed hash, you never know.

I don't know if it works or not, Just wanted to know for safety reason

like an attacker could do this during hard fork, simply to create mess

as miners need to switch from old algorithm to randomx

and all of sudden reorg happens

and miner need to switch randomx to cryptonight

as chain moved few blocks back

Yes, it should be possible.

vtnerd: I was looking into msgpack since you mentioned using it, is there a specific reason we'd choose this over, say, capnproto?

Btw, when Monero was switching from cryptonight to RandomX. Did it change the difficulty by hardcoding it?

No.

Didn't you faced block time issue, because difficulty in cryptonight when you had 600 Mh/s and all of sudden you switched to RandomX, and that difficulty could be way more for you compared to cryptonight

Most miners select PoW algorithm from the block verison, so switxch back and forth ought to be fine fwiw

I don't think so.

Oh ok thanks for your input

It wasn't much different. Like x3 or so ?

Well just think it like this

1000 Rx VEGA 64 are mining Cryptonight and all of sudden you switch to RandomX, now those 1000 Rx VEGA 64 hash power decreases down to more than 3x in randomx algorithm

The main differences afaict, besides performance, is that capnproto uses a schema, where msgpack is schemaless.

This of course has obvious up- and down-sides. I think having an explicit schema is useful because it serves as a kind of contract for what gets sent over the wire.

Well that's the other end of the spectrum, certainly. Point is, we currently _do_ have a schema (which is enforced during deserialization), only the serialization protocol we use (json) does not support it

Using a protocol that _does_ support an explicit schema allows us to do away with a lot of custom "check if this schemaless thing has x and if it is of type y and if so copy it to member z" code

json does have a schema, most people don't use it

part of my investigation was seeing whether I could get the code to spit out a schema

message pack doesn't define a schema language, but its similar enough to JSON that it might be able to borrow its schema

capnproto and protobuf are both _not_ self describing. you cannot spit out the fields of the structs/objects. If message pack used integer keys (higher performance / wire size) its self describing, although possibly less useful since the programming doesn't automaticlaly know what "field 1" is in isolation

*cannot spit out the fields without the schema

*progamming doesn't automatically -> programmer doesn't automatically

the encoding scheme of cap n proto is a more complex, which is the primary reason I was iffy on it

Those are valid concerns, especially the "self describing" part

calculating the field offsets for a struct apparently are : "whatever the compiler does" ... Im sure its an efficient algorithm for space, but it creates a frustrating dependency

It's an extra dependency, to be sure. I wonder what it would do for build times, though. We have an extra step - which takes some time - but then you have simple, separate structs where you can include only what you need.

That in itself means you take a lot of serialization stuff which is now in headers out of the headers

Though I also saw you mention that you have already moved some of that serialization out of the headers so I'm not sure how much of a difference it would make there.

the structs encoding is not simple though

an encoder/decoder should be many more lines of code

and yes, switching to cap n proto to improve compile times is not a benefit when we could alter our JSON usage

as an example, the HTTP JSON-RPC uses epee for json. and its more straightforward to split the header/cpp for that since it only uses one DOM type. `src/net/tor_address.[h|cpp]` split it for instance

vtnerd: woodser: do what you want with spamming diffs and moving stuff across files. Looks like I'm not doing as much as I used to on monero, so others' preferences can now take over.

endogenic: that also goes for you, if you want to rewrite things.

lol

<woodser> :(

Sorry. Maybe I should have said that a couple days ago.

<woodser> if you have time to review the rest of wallet2 in #6332, it would be most appreciated. the changes permit direct import and export of wallet data for when the file system is not accessible

Yes, I'll do that soonish.

<woodser> thank you

vtnerd: i made a couple of testnet transactions and added net.p2p.tx:DEBUG. I get "Failed to get tx meta from txpool" in both cases

that's the only output i get in the log at least.

same for rbrunner: monero-project/monero #6314#issuecomment-591014738