00:16:59 The PR's on hold till someone with domain experience reviews sekreta really. 00:17:44 Though atm it seems to be a direct API for API translation layer rather htan the multiplexer I had been expecting. 00:23:50 i could review it 00:24:05 and give a report 00:24:39 I have understanding of I2P although the monero side of things needs some downloading 00:25:09 not sure about sekreta...I could review but it I think anonimal might be deeper into it than I am 00:25:40 i've been more focused on using available networks together intelligently instead of trying to improve on them 00:26:02 zzz of I2P would be an awesome person to have review it...for the I2P portion 00:26:12 or str4d 00:27:18 neither of them do c++ (at least to the extent demanded) 00:29:44 oh right, they're java guys 00:29:54 and i'm mainly a java guy too 00:30:22 (str4d is also fantastic at rust) 00:30:56 i've been tempted many times to go back to c++ but I think rust might be what I would move into...but not sure...c++ will be here forever i think, not sure about rust 00:32:01 well, we could review it as best as we could and just give a report on our thoughts...maybe those with people who are heavier in c++ could be enough? 00:33:01 Would it hurt to run it on testnet for awhile and verify? 00:33:33 What would be helpful is not so much a C++ review rather than a review of the higher level concepts. Like, does this break any of the protections i2p or tor provide, etc. 00:33:57 "this" being the intended implementation of the multiplexer layer, which is what makes it interesting. 00:34:24 Does it do what it intends to do, in theory. 00:34:37 Does it actually improve on just using, say, tor alone. 00:35:11 yeah it's a bit tougher to review a complete re-write of something versus just embedding something already written...you've got to really understand TOR and I2P to provide a real review 00:35:34 Right. That's what I meant by "someone with domain experience". 00:35:34 And that would be best IMO from someone like zzz and str4d 00:36:11 Or someone from zcash maybe. They could benefit from it (and us from the added traffic too). 00:36:22 I have the Java I2P working and can verify it, but verifying a rewrite... the original developers are best...well, who knows who jrandom is 00:36:23 I think they have at least one network person. 00:37:42 and a lead dev from torproject? 00:41:34 There's a guy from Nym who might be helpful too. He's been trying to rewrite I2P to include tokenization. 00:41:52 He's in the Bisq community sometimes. 03:48:52 After reading the documentation on Sekreta and reflecting back on 1M5, where I see convergence is on the idea of making anonymity networks easier to integrate. Both projects diverge on the reason why though. Sekreta focuses on an assumption that TOR/I2P and other anonymity networks are vulnerable to being 'broken' into and so why trust just one network whereas 1M5 focuses on multiple networks in case one is blocked at 03:48:52 a node, it can use other networks to route around the block. 03:50:30 Sekreta seems pretty low-level in that it is written in C++ of course, but also in that it seems to be focused on improving lessons learned with TOR/I2P, etc. and is trying to create a better experience and more secure multi-anonymity-network as if it was really one. 03:51:22 1M5 is more of a higher layer application router implementing a 'micro-service' event-driven architecture typically found in enterprise application type development. 03:52:29 It obviously comes out of the experience of the author's in integrating multiple technologies to innovate those into a larger web to build something new from a wide array of parts. 03:54:05 It focuses more on leveraging what's already in use in an innovative manner, e.g. using TOR when I2P is blocked to route around the block, and I2P when TOR is blocked...using Bluetooth when a government shutsdown cell towers nearby when activists are demonstrating. 03:54:21 Switching to long-range radio when bluetooth/wifi-direct gets jammed. 03:54:50 Switching to LiFi (Light Fidelity) to get information out of a zone being blanketed with radio jamming. 03:55:44 Their goals are both similar in making it easier to use current networks yet they diverge I believe in how they're likely to be used...who the targe audiences is. 03:56:15 I see them as complementary. 03:58:36 If we can get some devs like zzz/str4d or others the community agrees with to review Sekreta, I will look into what value Sekreta might bring to 1M5 in replacing the TOR/I2P implementation within it. This would delegate the lower-level anonymity network implementations to Sekreta while allowing 1M5 to focus more on integration with Bluetooth, WiFi-Direct, longer range radios (e.g. Locha Mesh & GNU Radio SDRs), and LiFi. 03:59:12 So in the mean time, we could provide multiple methods of proxying to Monero for testing out. 03:59:47 The --txproxy options could be TOR, I2P, Sekreta, 1M5, LOCHA 04:01:10 1M5 would seek to encompass all as that's it's primary motivation....to leverage the best networks to prevent blockage 04:01:42 It seems like Monero is in a holding pattern with 04:01:57 ...Sekreta and Kovri until they can be properly reviewed 04:02:24 What's implemented in 1M5 currently is TOR and I2P and now in progress is Bluetooth. 04:02:44 These are the standard TOR and Java I2P networks so nothing new. 04:02:48 nothing to test 04:03:28 The only difference is that it will use an appropriate network based on where the individual is. 04:03:47 It doesn't broadcast too all of its 8 nodes on all available networks. 04:04:56 It selects the appropriate network based on the user's input or the developer's setup if the GUI can not take into consideration the User input initially...GEO IP can also be used to set it up, similar to what I2P does when deciding not to publish IP addresses to its netdb 04:05:32 For example, while in China, it is likely best not to even attempt to use TOR behind any Chinese address 04:05:42 Use I2P by default 04:06:42 And have bluetooth working so that the chinese can help each other share access 04:08:01 This leads me to one thing about Sekreta in that combining multiple networks, it would need the ability to select which networks can be used or not used by the end user based on their situational awareness. 04:12:41 Anyways, it was great talking with anonimal and I'm hoping we can help make monero more censorship-resistance. 04:13:18 I think Venezuela would be an excellent testing ground...would love to see that out ASAP 04:43:12 monerod used 6400MB of ram today. 04:43:22 Had quite a few odd nodes as well connect. 06:06:19 martijn : tewinget added 0mq, but it required a few tweaks to get it working - specifically for the light-wallet-server. the RPC code should be working now 06:07:27 I have a series of patches to improve the performance of writing to json, as 0mq pub/sub is something I'm targeting to get added 06:08:09 _most_ of the patchs are PRed right now, with one other on hold for now. I'm also investigating msgpack performance, and finally found a suitable way to integrate that 06:08:36 but trying to get 0mq pub/sub integrating should come before any msgpack 06:11:00 one of the objectives is to minimize blocking of p2p i/o threads 06:11:27 although monero tx volume isn't substantial :/ 10:17:50 vtnerd: Those are interesting developments. pub/sub would be a very useful addition. Do I understand correctly that json rpc is now only implemented on the node side and that the wallet does not yet use it? Is that something you are actively working on or could help with? 14:20:52 the light-wallet-server uses it but not that wallet 14:21:36 I'm not considering changing the wallet; it uses a custom binary protocol that gives it some speed advantages 14:21:55 but it has HTTP overheader, so yeah its a tough one to "clock" 14:22:14 So you're going to skip the http-to-zmq and go right to the pub/sub for the wallet then? 14:22:33 it'll be easier to compre light-wallet-server pulling down txes and the wallet at some point, which won't be identical but close enough to give an indication 14:23:25 I haven't thought about adding pub/sub to the wallet at all 14:24:08 there are downstream people using wallet2 in android for instance, so its going to require some kind of fallback or something 14:24:18 Ah, well then I'm confused. Where exactly do we want to add pub/sub? 14:24:37 light-wallet-server wil be the testing ground since its already using zmq 14:24:54 and there are other people that have asked for it 14:25:08 another big one would be xmrig 14:25:41 you have to poll right now if the pool and node are't on the same box 14:25:48 and even then its triggering a process 14:26:53 I assume that means you lose efficiency due to the miner continuing to mine on the old block before it realizes a new block is there 14:41:02 yeah a bit of time loss, and increased polling uses more cpu 15:19:51 What is the re-org limit on Monero? 15:20:44 What is a re-rg limit ? 15:20:57 I mean the maximum block height upto which fork is allowed? 15:21:58 Hmm. It should work fine till the last checkpoint. Then, it should work fine before, but I don't think that's been tested. 15:22:20 (feel free to, and report a bug if it fails :)) 15:22:58 It might require qiute a lot of memory if you reorg hundred of thousands of blocks though, so some nodes might croak. 15:23:39 When monero was hard forked to RandomX, lets say at block height 10000, is there a possibility that someone fork the node and move back to block height 9500? Or when you hard fork , you don't allow re-org beyond hard fork block height. 15:24:02 What does forking a node mean ? 15:24:25 I mean mining more difficult chain, which is not the part of main chain. 15:24:44 So someone starts mining again from 9500 onwards ? 15:25:04 That should work fine, if they prevent reorg to the longer chain. 15:25:20 no, lets say someone mined from block 9000 to 9500, whose chain has more difficult than main chain at height 10,000 15:25:24 If they end up with a longer chain at some point, other nodes will reorg to it. 15:25:51 so in case of monero, reorg happens based on longer chain? 15:25:52 Well, nodes reorg to the longest valid chain they know of. 15:25:59 oh 15:26:08 ok thanks 15:26:10 long here meaning most cumulative work, not height. 15:26:15 yes 15:26:35 I mean the same when I say most cumulative work for someone mined 9000 to 9500 is more than the main chain at 10,000 15:26:47 I've tested reogr past a fork before. I don't think I did with the randomx fork itself. 15:27:16 But I don't see why it would not work. Possibly something to do with the seed hash, you never know. 15:27:32 I don't know if it works or not, Just wanted to know for safety reason 15:27:44 like an attacker could do this during hard fork, simply to create mess 15:27:54 as miners need to switch from old algorithm to randomx 15:27:58 and all of sudden reorg happens 15:28:06 and miner need to switch randomx to cryptonight 15:28:10 as chain moved few blocks back 15:28:20 Yes, it should be possible. 15:28:43 vtnerd: I was looking into msgpack since you mentioned using it, is there a specific reason we'd choose this over, say, capnproto? 15:29:08 Btw, when Monero was switching from cryptonight to RandomX. Did it change the difficulty by hardcoding it? 15:29:42 No. 15:30:40 Didn't you faced block time issue, because difficulty in cryptonight when you had 600 Mh/s and all of sudden you switched to RandomX, and that difficulty could be way more for you compared to cryptonight 15:30:42 Most miners select PoW algorithm from the block verison, so switxch back and forth ought to be fine fwiw 15:30:55 I don't think so. 15:31:04 Oh ok thanks for your input 15:31:07 It wasn't much different. Like x3 or so ? 15:31:15 Well just think it like this 15:31:18 * moneromooo afk for a bit 15:32:16 1000 Rx VEGA 64 are mining Cryptonight and all of sudden you switch to RandomX, now those 1000 Rx VEGA 64 hash power decreases down to more than 3x in randomx algorithm 15:34:05 The main differences afaict, besides performance, is that capnproto uses a schema, where msgpack is schemaless. 15:35:24 This of course has obvious up- and down-sides. I think having an explicit schema is useful because it serves as a kind of contract for what gets sent over the wire. 15:42:23 * asymptotically dusts off the ASN.1 compiler 15:53:05 Well that's the other end of the spectrum, certainly. Point is, we currently _do_ have a schema (which is enforced during deserialization), only the serialization protocol we use (json) does not support it 15:54:54 Using a protocol that _does_ support an explicit schema allows us to do away with a lot of custom "check if this schemaless thing has x and if it is of type y and if so copy it to member z" code 16:04:27 json does have a schema, most people don't use it 16:05:06 part of my investigation was seeing whether I could get the code to spit out a schema 16:05:53 message pack doesn't define a schema language, but its similar enough to JSON that it might be able to borrow its schema 16:07:38 capnproto and protobuf are both _not_ self describing. you cannot spit out the fields of the structs/objects. If message pack used integer keys (higher performance / wire size) its self describing, although possibly less useful since the programming doesn't automaticlaly know what "field 1" is in isolation 16:07:52 *cannot spit out the fields without the schema 16:08:17 *progamming doesn't automatically -> programmer doesn't automatically 16:08:48 the encoding scheme of cap n proto is a more complex, which is the primary reason I was iffy on it 16:10:07 Those are valid concerns, especially the "self describing" part 16:10:31 calculating the field offsets for a struct apparently are : "whatever the compiler does" ... Im sure its an efficient algorithm for space, but it creates a frustrating dependency 16:17:36 It's an extra dependency, to be sure. I wonder what it would do for build times, though. We have an extra step - which takes some time - but then you have simple, separate structs where you can include only what you need. 16:17:52 That in itself means you take a lot of serialization stuff which is now in headers out of the headers 16:18:49 Though I also saw you mention that you have already moved some of that serialization out of the headers so I'm not sure how much of a difference it would make there. 16:24:00 the structs encoding is not simple though 16:24:29 an encoder/decoder should be many more lines of code 16:25:24 and yes, switching to cap n proto to improve compile times is not a benefit when we could alter our JSON usage 16:26:51 as an example, the HTTP JSON-RPC uses epee for json. and its more straightforward to split the header/cpp for that since it only uses one DOM type. `src/net/tor_address.[h|cpp]` split it for instance 16:28:42 vtnerd: woodser: do what you want with spamming diffs and moving stuff across files. Looks like I'm not doing as much as I used to on monero, so others' preferences can now take over. 16:29:12 endogenic: that also goes for you, if you want to rewrite things. 16:29:46 lol 16:33:44 :( 16:34:38 Sorry. Maybe I should have said that a couple days ago. 17:04:22 if you have time to review the rest of wallet2 in #6332, it would be most appreciated. the changes permit direct import and export of wallet data for when the file system is not accessible 18:10:37 Yes, I'll do that soonish. 18:24:50 thank you 18:56:55 vtnerd: i made a couple of testnet transactions and added net.p2p.tx:DEBUG. I get "Failed to get tx meta from txpool" in both cases 19:07:46 that's the only output i get in the log at least. 19:35:17 same for rbrunner: https://github.com/monero-project/monero/pull/6314#issuecomment-591014738