hyc: if I call a loop of m_db->have_block(H); sleep(1) in a thread, without an overarching db txn, and at some point another thread adds the block H, which did not exist prior to the addition, will have_block switch from returning false to true at roughly the right time ?

If sounds "yes, duh", but it's unclear when a read txn "resets".

if there's no overarching txn, then yes, that call gets a new txn each time

and a new txn always sees the current state of the DB at its time of creation

ty

so, yes, duh. :P

i first started monero as a gui and clicked on remote node. but now I started monerod to run a fullnode myself. will my gui recognize it when restarting or do I have to reconfigure it?

FROMSOUTHWEST: select local node inside Settings -> Node

ok, thx

.merge+ 7234 7235 7220 7249 7246 7247 7248

Added

.merge+ 7250

Added

vtnerd: please also 7251 against master so that we don’t go out of sync

.merge+ 7251

Added

Is there a limit on number of i.p. addresses you can have in the ban-list for monerod?

Based on my limited experience working with the JsonRPC, I'd say No. `std::map<std::string, time_t> blocked_hosts = m_p2p.get_blocked_hosts();` Blocked hosts are stored in a map that can get as large as you'd like it to be

I've got 2833 addresses blocked, including all TOR exit nodes on my two full nodes to see if it stops the nonsense.

my node did not crash once with gui.xmr.pm/files/block_tor_new.txt

I'm not accepting incoming connections :D

that works too lol

There are a number of i.p. addresses on your list that are not on mine. I'm going to combine both lists, get rid of the duplicates and use it.

mine is tor exit nodes + relay nodes with 18080 exit policy

What's the ethos on allowing people to make mistakes? For example, in my API I can theoretically allow anyone to send an unlock_time of up to uint64_t. But that unlock_time (the largest unsigned 64 bit integer) is literally an eternity. Should I cap the unlock_time value at 1 year's worth of blocks? Say, throw an exception? In C# there is no

overflow checking, so if someone does 0 - 1, they'll get ulong.MaxValue (18,446,744,073,709,551,615), which is a long time to be waiting for your funds to unlock :)

My list is a complete listing of all tor exit nodes. I'm missing your relay nodes . Combining and getting rid of duplicates will add them in.

Mmmmmmmmmm: github.com/monero-project/monero#blockchain-based

I don’t think other APIs limit this

If someone accidentally misuses the API and sends funds with an unlock time of 18 quadrillion blocks, I'm getting a hitman called on me.

lol

Maybe I'll add a Safe and Unsafe transfer mode

Imagine Hodling for 18 Quadrillion blocks...

diamond hands

jess: I can tell you why you can't defeat the 'spam'. You are thinking in cult doctrine. If it was real spam, and I was selling Viagra for example - you could easily ban keywords and urls. Instead, stop being a sheep, think like a cult leader. Recoginse that this 'spam' is just some bullshit that you tell to the sheep.

how can the miner retrieve the fee when the amount of the outputs being send is hidden?

Go through all the txes included in a block, sum up the fees, add that to the subsidy.

Output amounts being hidden is irrelevant.

does the miner calculate the fee of every transaction himself?

sry im a bit confused

FROMSOUTHWEST the tx fee is in plaintext in the transaction. The output commitments only commit to 0 with the fee taken into account.

ah ok. so i just subtract the fee amount from my change?

If you send X, using Y, and the fee is F, you will get back Y-X-F as change.

ok, thx

is the blockreward penalty only when the block size is bigger than 300kb or also when the block the block size is bigger than the average of the last 100 blocks?

Bigger than the median of the last N blocks, possibly with a delay.

Hi and welcome in the New Year.

My browser crashed. Was there any feedback on my initial ML attack detection proposal? If yes, please kindly copy-paste it. (I need to install that bouncer today)

my node is currently unresponsive to RPC, but isn't using more ram or cpu than normal. is there like, a new fun thing going on?

@Lyza is monerod process in a running state? Is the machine as a whole otherwise responsive?

<dgoddard> yes

just restarted and it looks like I'm about 700 blocks /24 hours behind

Lyza: which version are you running?

0.17.1.8

dns blocklist enabled?

yes and also using block_tor_new.txt

ok, DNS blocklist has a deadlock

which Ik is redundant but I was still getting oom errors without the tor stuff blocked

that can trigger if you are unlucky

oh ew ok will disable for now then

it is fixed in code already

cool

will be included in the next release

perfect, looking forward to being able to use that and stopblocking tor exit nodes, even though people shouldn't be using exit nodes to connect to me =P

@selta was this the fix? monero-project/monero #7221

for what?

deadlock? monero-project/monero #7239

ty

can someone explain me what deadlock banning is

it means the daemon freezes if it tries to ban peers while it is currently updating the peer list

just a bug, not an attack

it also happens only rarely

.9 tag when?

soon^tm, still trying to set the object limit so that it does not break wallet synv

xync

.merges

calling monerod ban with a ban list spams the absolute hell out of the log file if it's a long list like block_tor>new.txt -- just saying

spams 1000 entries?

or more than 1 per ip?

1 per ip

but I don't care, my bitmonero.log is already 45 MB :D

not sure how else to do this apart from printing the IPs that are getting banned

yeah is only 1 per entry but still, would be nicer I think to just say like 'loading file whatever.txt into ban list' or w/e

just makes it annoying to look for anything else in the logs, my logs are like 98% so and so is blocked because I update the ban list once an hour

man 1 grep

lol thanks yeah I can use grep, could probably get better at it. still though.

*shrug* I ain't arguin it was just a thought. seems a bit much to me

which log level do you use?

the default, zero

honestly it's ban enough that I'm going to go back to downloading the new list, diffing them, and banning the new entries one at a time. the way @ban works now, it takes a full ~20 seconds to run as it appears to load the every entry in the list one at a time, every time

you mean the ban @filename command?

yes

ok, weird

it takes less than 1 second on my system

weird

ssd?

nah it's a spinning raid5

maybe that’s why

not sure why it would cause that much disk io

ok looking at older logs, it did load faster when I wasn't actively syncing, I was still finishing up from being a day behind

with .9 hopefully the ban list will be less important again

hopefully :)

selsta do you know if the monerod freezing issue also happens with the ban command, or just the dns based bans?

both

cause my RPC seems frozen again, even though I'm synced with the network

ah shit

well damn

hmm

if you only apply the ban list on start

it should not happen

with the next version it can't freeze anymore

roger that I'll just restart monerod to update the ban list for now, and only update it like once a day

the longer the ban list the higher the chance for it to freeze

makes sense, unfortunately I was still crashing from OOM without blocking the tor nodes

.merge- 7248

Removed

^ readded once confirmed

.merges

Snipa luigi1111w? :)

moneromooo: I assume monero-project/monero #7254 will result in less bans, right?

due to them being deprioritized after kicking once

Likely.

why I got so little connected incoming peers after switching to 0.17.1.8, maybe i messed up my firewall settings or new nodes cant connect to old nodes and vice versa?

how many do you have?

did you just freshly start your node?

yeah i think im getting more, but I used to get more new connection quicker after freshly starting my node, but its ok i guess

mine have between 40-70 incoming connections which seems normal on v0.17.1.8

I got 9 incoming peers after 15mins FYI

that sounds normal

I'll ask my question in a different way: If I spend time on properly analysing the banning problem from a ML perspective, create a proper CCS proposal, and in the end provide an elegant self-adapting solution, would you be fine accepting it or are you in a way ML averse? Or have some questions that need clarifying?

How good it is, whether it can adapt well (ie, no overfitting to current weather), whether it needs gobs of CPU, how much new code it needs...

Who knows in advance without more info ?

Oh, and how much type 1/2 errors.

Guess it's how good it is.

OK. You're not ML averse, just skeptical. This is OK for me.

So these questions I will be able to answer once I'm nearly ready with this. So just to mention that if I go in this direction (which is a large investment), and I provide satisfying results, I understand that you'd accept it.

as far as I understand.

I'll refer you to my previous line :)

There are methods to make it good. It requires time.

And imagine a situation that I work like a dog, I'm satisfied and can prove that it's good and somebody tells me - not acceptable, because Skynet, I'd be quite sad :)

Overfitting can be mitigated by simulating various random time series processes. I imagine, that I'd define a few bad agents, which after a period of silence, would suddenly start spamming. The quality would be measured by the amount of 1/2 errors.

*few bad agents and many good agents of course.

And by saying "time series process" I mean for instance the distribution of traffic over time.

That kinda screams for a proxy, does it not.

Then you can go wild.

Proxy in what meaning? I imagine that each node would have a pre-trained model and would in parallel do an on-line training, and perhaps make an ensemble method of both.

Proxy being MITM software.

I don't understand where MITM software fits to what I'm saying.

I come from a bit different field :)

Software that runs between monerod and the network.

Which then decides whether allow some traffic or not.

OK I'd say that the monerod would just have a class that does it.

No need to making an another binary, from my perspective.

I just need to hook some filter into the part, which has access to realtime traffic flow, nothing more.

.merges

.soon

I think this makes an easier architecture, but I'm open for other opinions.

Can somebody tell me how you extend the ban list currently? By hand? Is there a class that tries to do it automatically?

Call block_host on the p2p object.

I shall have a look. Thanks.

And to make it clear - the fitness will strongly depend on the quality of the simulation. I just think, that it's quite easy to define the objective function here by answering the question "what hurts me most?". I currently imagine that it's sustained high traffic from a given host or from a set of hosts, incomparable to the traffic of other connected hosts (even if they're in minority)

I think the objective is wrong. That can be left to existing software. If you want to do some ML based analyzer, I think you should concentrate on monero specific things, not things that any existing proxy can handle.

(I assume there's already lots of existing proxies to detect/shed high traffic, but I don't know that for a fact)

I'm not pushing a specific solution.

It's just my idea how it could be handled.

My current idea.

Or in general I'm trying to address the problems, currently plaguing the development, whatever solution we choose.

I'm just reading up on the proxies.

It just means a separate binary, without having to plug it in another.

I understand now. I'm just looking for a proxy solution, being able to filter out spam.

I think it would make more sense for someone who has a let of network knowledge to look at our current code and search for issues / improvements, a ML solution sounds significantly more risky, especially if the underlying code still has issues.

I'm not a network specialist. You only have to take into account, that for Linux you'll find filtering tools, but Monero is multiplatform, so no IPTables under Windows.

do you mean no iptables, or nothing like it ? The former, maybe, but I have trouble believing you if the latter.

Then again I really don't care to discuss windows firewalls ^_^

I mean no iptables. The reason why I'm saying this, is that you don't want to have to maintain a different proxy for each OS. I'm just looking for something multiplatform.

I already figured out that for the testing part (not production), one could use several traffic generators.

luigi1111w: how soon is soon

Not sure about the actual filtering.

I see, that there are some GUI tools, but neither they are multiplatform, nor free / open source/

For limiting bandwidth.

tbh I'm not sure there'll be many people who (1) run monerod on more than one OS and (2) don't already have their preferred firewall.

So making yet another firewall just for this is likely not very productive.

Do you suggest then rather creating a proper firewall configuration?

No.

and distributing it? Or at least documenting?

And if a node of a Windows user doesn't actively block the bad traffic, but Linux do, because we've setup a firewall (somehow) under Linux, you'll be effectively locking out Windows users, since they will be spammed.

tonight probably

It might be maybe 500 lines of portable C++ code. And by saying ML, I'm not necessarily pointing at Deep Learning stuff. Sometimes it's enough just to use a moving average and a moving standard deviations.

But OK. If you change your mind, you know whom to ping.

That seems ok to have in monerod. I was assuming machine learning since you mentioned it.

Even the linear regression is a machine learning method. So sorry for confusion.

Here's what I propose - I'll try to create a very rough proof of concept, if *anything* works under a poor-man's simulation of an attack, I'll create a proper CCS proposal with the initial results, and we shall talk later. Deal?

(and the simulation will be done via nmap)

500 lines of C++ for things like linear regression for monerod specific stuff seems ok. The same for traffic volume/endpoints doesn't IMHO. But I don't get to decide so don't feel like you have to get a deal with me.

Once a traffic volume DoS hits monerod, it's a bit late to do anything anyway.

Sure, I'm waiting for feedback from others.

iptables or wahtever else will be a lot faster to drop.

monerod has access to info iptables (or similar) does not, so that's what it should be using to determine whether some traffic is bad.

Couldn't monerod expose an ip to iptables or other FW right after detection?

Sure. Sounds good.

I'd suggest using the same scheme as currently ("dropping connection$" on some particular category).

Noted down.

nmap has got it all... null-byte.wonderhowto.com/how-to/us…launch-dos-attacks-and-more-0168788

Snipa: do you have a bit earlier time for merges than luigi?

Something is conflicting so we need some merges.