07:19:27 <mj-xmr> utxobr[m], I wrote my comment. Please don't mind the dose of honesty.
08:31:16 <selsta> hyc: can you try to reproduce the issue with syncing off one exclusive node with --limit 999999999 added to both nodes?
08:31:32 <selsta> so that we can make sure that it isn't some bandwidth limit thing
09:08:47 <hyc> er.... why would it be a bandwidth thing between 2 nodes on a LAN not talking to anyone else?
09:20:06 <wfaressuissia[m]> software bandwidth limit is too small and can be reached in case of fast hardware of peers and fast hw connection between them
09:20:28 <wfaressuissia[m]> But it can be wrong assumption too
09:26:05 <hyc> we have never changed the default limits and node-to-node syncused to work fine
09:26:41 <hyc> I routinely sync'd two nodes from each other on localhost without issues, and if bandwidth limit was going to hit anywhere, it would be there
09:26:42 <moneromooo> Sync is broken now ?
09:26:53 <hyc> I mean IBD
09:28:28 <hyc> cannot get a successful sync of a fresh node from an existing node over LAN with --add-exclusive-node all in one go
09:28:48 <hyc> it always stops and bans the existing node.
09:29:02 <hyc> will resume if you manually unban or restart the new node
09:35:57 <hyc> eh what happened to external/miniupnp in master? trying to update to a nonexistant commit hash
09:38:42 <hyc> oh, origin changed
09:42:30 <moneromooo> git submodule sync
09:42:51 <moneromooo> Does it ban soon or after a certain height only ?
09:43:07 <hyc> seems to be random time intervals, not particular heights
09:43:28 <hyc> hang on, rebuilding current master, will spin up a new localhost-only test
09:50:42 <hyc> conn breaks on localhost too
09:50:57 <hyc> 1st instance at height 19456
09:51:25 <hyc> it reconnected about 1.2sec later and resumed
09:51:42 <hyc> disconn again at height 29184
09:51:44 <moneromooo> Oh it's drop, not ban ?
09:51:50 <hyc> it will eventually ban
09:51:57 <moneromooo> OK. I'll check later today.
09:52:12 <hyc> disconn again at 38912
09:52:24 <hyc> 48640
09:52:44 <hyc> 58368
09:52:46 <hyc> you get the idea
09:52:49 <wfaressuissia[m]> hyc, Can you post +-few minutes log around disconnection event from both nodes ?
09:53:18 <hyc> I have posted these before, but hang on
09:53:40 <wfaressuissia[m]> the last log was only for one node and had not very verbose log level
09:54:01 <hyc> not sure I'll see the disconnect event at this loglevel
09:55:42 <hyc> ok, have an event captured
09:59:17 <hyc> hm, nothing interesting happened on the other side. lemme try again
10:03:04 <wfaressuissia[m]> there must be something interesting :D, keep going
10:03:57 <hyc> ok logs uploaded
10:04:29 <hyc> http://highlandsun.com/hyc/log1.txt and log2.txt
10:04:36 <hyc> log1 is the fresh node
10:04:52 <hyc> it's captured from the time of one "disconnected" message to the time of another
10:05:03 <hyc> log2 is the existing node from the same timespan
10:05:14 <hyc> both at loglevel 2
10:06:47 <hyc> so far no bans
10:08:25 <hyc> hmm, "We can download nothing from this peer" ??
10:08:47 <hyc> 10:00:39.937
10:08:49 <moneromooo> That is not a ban inducing drop. It's the caller deciding not to continue.
10:09:10 <moneromooo> So if you get banned, there must be another type of drop.
10:11:32 <hyc> even so, that sounds incorrect. certainly the peer has the entire blockchain left to send
10:12:16 <moneromooo> Sure, it could be made to be more precise. Not really relevant though, it'll reconnect.
10:13:30 <hyc> so so far it's just that, no ban on localhost
10:13:49 <hyc> I guess I'll have to try it again over LAN, that's where bans showed up last time
10:15:46 <hyc> can we raise loglevel to logfile without raising level on console?
10:16:42 <moneromooo> Not without code changes.
10:17:08 <moneromooo> Though --detach might help ?
10:17:34 <hyc> I just want to see the normal sync Info messages on console
10:17:49 <hyc> and the disconnect/ban messages
10:18:09 <selsta> there is some log category magic mooo knows :D
10:18:11 <moneromooo> tail -f bitmonero.log | tee log | grep --lined-buffered Sync :P
10:18:44 <hyc> hmph
10:19:01 <hyc> I guess that'll work
10:27:52 <hyc> ok starting over with LAN instead
10:33:04 <hyc> tail -f doesn't work across log rotation, boo
10:34:24 <hyc> guess I'll just leave this for a few hours and see if it happens later
10:36:59 <wfaressuissia[m]> `tail -F`
10:37:06 <wfaressuissia[m]> works across log rotations
11:04:02 <wfaressuissia[m]> "hmm, "We can download nothing from this peer" ??" fresh node behaviour: 1. fetch hashes for the next 10000 blocks; 2. fetch these blocks; 3. drop connection since we've fetched all blocks from the peer
11:04:51 <wfaressuissia[m]> but it should be 3. request hashes for the next 10000 blocks if peer has superior advertised height and we've downloaded all previous blocks
11:06:07 <gingeropolous> id guess this behavior was implemented to get the syncing node to try and find other peers
11:08:04 <moneromooo> No, originally it'd do that but I added a number of checks for bad peers and one thing bad peers can do is send bad hash sets, and I think what happens might be that it can't tell between "I'm out of hashes from that peer but I did download stuff just fine" and "I'm out of hashes from that peer because all the ones I got were dud".
12:05:31 <gingeropolous> which would be a fine behavior unless --add-exclusive-node is being used
13:38:21 <hyc> right, with --add-exclusive-node I should never see the conn being dropped or banned. ideally.
13:39:02 <hyc> caught a weird timeout on LAN this time but it looks like the full node froze for a minute.
13:39:10 <hyc> no log lines at all for 1 minute
13:39:36 <hyc> this is using the Mac as the full node this time, previously both daemons were on my linux box
13:40:54 <hyc> but yeah, now the new nde has banned the fullnode
13:51:55 <wfaressuissia[m]> "no log lines at all for 1 minute" with which log level ?
13:52:00 <hyc> level 2
13:52:27 <hyc> http://highlandsun.com/hyc/log3.txt and log4.txt
13:52:31 <hyc> log3 is the new node
13:52:35 <hyc> log4 is the full node
13:52:57 <hyc> freeze occurs after 10:41
13:53:15 <hyc> after that the new node bans the full node
13:56:22 <selsta> hyc: unrelated but did you compile for mac arm or is it translating x86_64?
13:56:31 <hyc> it's arm
14:16:59 <wfaressuissia[m]> "no log lines at all for 1 minute" it can be only due to reached upload software limit
14:17:30 <wfaressuissia[m]> Can you repeat with higher log level or set explicitly high --limit in order to exclude this variant ?
14:17:54 <wfaressuissia[m]> there must be something that explains unresponsiveness
14:20:10 <hyc> the mac is running as a normal node, so it should also be talking to other peers. also there is a miner pointed at it. it should be processing get_height RPCs once per second.
14:20:27 <hyc> none of that is occurring, so I think something else is going on inside the daemon at the time
14:20:40 <hyc> and RPCs are not bandwidth limited at all
14:24:50 <wfaressuissia[m]> anyway we need more info, so it's either higher log level or debug build under gdb or something between
14:25:20 <wfaressuissia[m]> "none of that is occurring" yes, it looks like so
14:25:50 <hyc> can't catch it in the act though, so gdb is unlikely
14:27:15 <wfaressuissia[m]> `while true; do curl -m 30 http://127.0.0.1:18081/getheight || kill -STOP $(pgrep -n monerod) && break; done;`
14:27:25 <wfaressuissia[m]> it should catch unresponsiveness
14:27:47 <wfaressuissia[m]>  * `while true; do curl -m 30 http://127.0.0.1:18081/getheight || kill -STOP $(pgrep -n monerod) && break; sleep 1; done;`
16:57:43 <hyc> hm, ok maybe throttle related
16:58:13 <hyc> http://highlandsun.com/hyc/log5.txt and log6.txt
16:58:20 <hyc> using loglevel 3 on the fullnode
16:58:41 <wfaressuissia[m]> good, one less mysterious problem
16:59:04 <hyc> loglevel 2 on the new node shows a gap from 16:46:20 to 16:46:44
17:04:34 <hyc> maybe some math overflow in the throttle sleep calc
17:04:59 <hyc> 2021-05-04 16:46:20.176 [P2P5]  TRACE   net.throttle    contrib/epee/src/network_throttle-detail.cpp:306    SLEEPdbg >>> global-OUT: speed is A=2.12089e+06 vs Max=2.09715e+06  so sleep: D=0.106989 sec E=19461286 (Enow=19494054) M=2.09715e+06 W=   9.176 R= -217819 Wgood      11 History: [0 4458618 3466595 5788042 4839289 417088 43 491611 0 0 ] m_last_sample_time=1.62015e+09
17:11:24 <hyc> dunno. still a gap in the fullnode log between 16:46:22.493 and 16:$7:13.732
17:11:31 <hyc> 16:47:13.732
17:16:44 * jbigs[m]  < https://matrix.org/_matrix/media/r0/download/matrix.org/xvbpnnDuklUXYsplgERSewCJ/message.txt >
18:10:46 <gingeropolous> jbigs[m], your sorta in the right place. "I am looking for someone to point me in the right direction for where I could get involved with implementations, policy research, marketing, and sales opportunities with Monero. " ..
18:10:54 <gingeropolous> though id move this to #monero .
18:13:39 <moneromooo> Getting involved with implementation is here and github. Pick something that annoys you, get an ok in principle here, make a patch. That's a good way to start.
18:14:20 <gingeropolous> moneromooo, can you access those matrix.org pastes?
18:14:20 <moneromooo> We don't sell monero though, we're not a company. So we don't have a sales department or the like.
18:14:25 <moneromooo> Yes.
18:14:34 <moneromooo> I tend not to though.
18:14:37 <gingeropolous> k. didn't know if tor made it difficult etc
18:14:45 <moneromooo> No, oddly.
18:15:00 <moneromooo> So I hadn't seen that one :)
18:16:43 <moneromooo> For policy research, there's a channel called... #monero-policy IIRC. It's infuriating though, since the topic is often mass spying.
18:17:18 <moneromooo> There are a few people who do marketing, thunderosa, geonic are some. I think xmrhaelan[m]1 too ?
18:50:04 <xmrhaelan[m]1> jbigs see #monero-outreach
20:58:08 <Guest_AP> Hello, I am looking for someone to show me what missing tests that have to be written for this project or any tasks that needs to be worked on. I am interested in contributing to this open source project in hopes of bettering my skills in C++.
20:58:33 <moneromooo> Hi.
20:58:48 <moneromooo> Tests are always welcome, thank you :)
20:59:16 <moneromooo> I don't have a list handy, but there are several types of tests we have:
20:59:56 <moneromooo> unit tests basically test a particular function: if you can isolate a function which doens't need too much state, adding quick runtime unit tests for it is always useful
21:00:42 <moneromooo> core tests are for interacting with the chain, so for consensus positive/negative testing - like create a tx or block that's invalid in some way and checking it gets properly rejected
21:01:15 <moneromooo> functional tests are high level, using daemon/wallet RPC to ensure stuff works as it should, they're written in python
21:01:42 <moneromooo> These are the ones that are likely to be added to.
21:02:07 <moneromooo> Core tests are notoriously hard to write (but when you do write one, they're damn powerful).
21:02:43 <moneromooo> Writing tests typically requires good working knowledge of how stuff works first though.
21:03:15 <moneromooo> For unit tests, this is usually small enough, but core tests and functional tests require more general knowldge in order to make a useful test.
21:04:26 <moneromooo> For unit tests, some stuff that's currently internal might be exposable and tested. However, it the exposing part isn't trivial (ie, requires code surgery), it might not be worth the changes. Details in the devil.
21:05:01 <moneromooo> Maybe a good first port of call is to grep various RPC calls in tests/functional_tests and seeing if any aren't used/tested.
21:05:54 <moneromooo> Actually, you said C++ skills, functional tests are in python, so nevermind.
21:06:23 <moneromooo> So, unit tests, you could scour src/cryponote_basic/ to check self contained functions.
21:06:44 <moneromooo> Also src/common and contrib/epee.
21:08:01 <moneromooo> There are also fuzz tests, in C++. If you find a self contained function that parses/processes untrusted input, it would be a good candidate for a new fuzz test.
21:08:44 <Guest_AP> Thank you for the feedback! This is actually my first time engaging in an open source project so all this is information is pretty overwhelming and exciting at the same time,
21:11:17 <jbigs[m]> You picked the right community to work in
21:11:27 <jbigs[m]> In the crypto space, it doesn't get much better than the Monero community
21:12:35 <jbigs[m]> The entire currency is a testament to freedom, collaboration, and the protection of privacy and liberty
21:12:40 <jbigs[m]> No other project comes close
21:16:49 <Guest_AP> Sounds good! (y) :]
22:35:24 <selsta> .merges
22:35:24 -xmr-pr- 7349
22:39:26 <selsta> .merge+ 7667 7663 7664 7665 7666 7670 7668 7677 7678 7680 7681 7682 7685 7695
22:39:27 <xmr-pr> Added