00:21:08 <cornfeedhobo> cankerwort[m]: no
04:29:14 <usamat> hello.
04:29:15 <usamat> tx JSON: https://termbin.com/ypkk
04:29:15 <usamat> The CLSAG element is an array. It has only 1 index in this txn (e.g CLSAG[0])
04:29:16 <usamat> When can there be multiple objects in the CLSAG array?
05:42:01 <usamat> anyone?
09:08:03 <mj-xmr> luigi1111w, Thanks anyway for merging my CCS. This will help in a different dimension.
12:53:52 <marcus40> Is Monero's current LMDB configured to allow duplicate keys with different values?
12:54:38 <moneromooo> Depends which table.
12:58:21 <marcus40> Where is the table-specific setting for that in the code?
12:58:38 <marcus40> or is it just enforced by flags in the  mdb_cursor_put calls?
12:59:15 <marcus40> I see this for example:
12:59:16 <marcus40> #define MDB_APPEND	0x20000
12:59:17 <marcus40> #define MDB_APPENDDUP	0x40000
13:00:28 <marcus40> why, for example, do we store the block info like
13:00:29 <marcus40> result = mdb_cursor_put(m_cur_block_info, (MDB_val *)&zerokval, &val, MDB_APPENDDUP);
13:00:50 <marcus40> when the description for appendup says:
13:01:01 <marcus40> Store multiple data items in one call. Only for #MDB_DUPFIXED.
13:03:03 <moneromooo> grep lmdb_db_open in src/blockchain_db/lmdb/db_lmdb.cpp
13:03:36 <marcus40> ah got you
13:03:41 <marcus40> so the flags are set there?
13:04:02 <moneromooo> The ones with DUPSORT allow duplicate keys.
13:05:04 <marcus40> thanks
13:05:22 <marcus40> what about storing the block info as i mentioned above
13:05:46 <marcus40> why MDB_APPENDDUP? The description says we're storing multiple data items, but it seems that we're only storing one piece
13:06:10 <moneromooo> Are you sure you're not thinking of MDB_MULTIPLE ?
13:07:21 <marcus40> ah my mistake
13:07:22 <marcus40> thanks
13:59:15 <Yonatan[m]> What's your guy's thoughts on users providing hashpower as form of payment for a product/service?
13:59:57 <Yonatan[m]> For example an online game that gives you in-game currency in the background in exchange for background mining
14:03:12 <marcus40> That is not an example of your question
14:03:35 <marcus40> the example has in game currency as the payment for the hashpower service
14:05:20 <marcus40> I guess if you partnered with a game creator, you could have something like that though
14:05:44 <marcus40> but few people would take hashpower as a payment
14:06:17 <marcus40> because essentially it's a gamble on future volatility
14:06:39 <marcus40> (of the price of whatever they're mining)
14:07:33 <marcus40> But I guess if they really wanted the ingame currency and there was no easy way to get hold of it they might overlook that
14:31:00 <moneromooo> I would like this to happen.
14:31:13 <moneromooo> If you want to do something about this, please feel free.
14:31:27 <hyc> hmmm. looks like randomx disabled_flags() isn't checked for FLAG_LARGE_PAGES
14:31:40 <marcus40> is mdb_set_dupsort   for sorting keys only?
14:31:44 <moneromooo> Two things of interest: primo and... not sure what it's called, tevador made a layer above randomx that a browser can call/use.
14:32:02 <hyc> tevador's randomx server, yah
14:32:46 <moneromooo> https://repo.getmonero.org/selene/primo for primo
14:33:24 <moneromooo> It works for apache (web server) and firefox, but could work with anything where you want payment in return for a service really.
15:22:35 <mj-xmr> An initial attempt to stabilize the functional_tests_rpc's mining test:
15:22:36 <mj-xmr> https://github.com/monero-project/monero/pull/7371
15:45:22 <selsta> mj-xmr: the problem is that it is difficult to test if this does indeed fix it
15:47:17 <marcus40> hyc do you know why for txs_prubable_hash we define the dupsort like:
15:47:18 <marcus40> mdb_set_dupsort(txn, m_txs_prunable_hash, compare_uint64);
15:47:39 <marcus40> because when we do a put, we're putting in a hash
15:47:40 <marcus40> mdb_cursor_put(m_cur_txs_prunable_hash, &val_tx_id, &val_prunable_hash, MDB_APPEND);
15:49:29 <hyc> I didn't write that code, dunno
15:49:50 <hyc> mooo did th pruning stuff
15:51:00 <hyc> marcus40: a val_tx_id is a 64bit uint
15:51:02 <hyc> not a hash
15:52:11 <hyc> I see, the key should be uint64, the hash should be compare_hash32
15:52:13 <hyc> could be a bug
15:53:40 <hyc> moneromooo: ^^
15:54:10 <hyc> fixing it would require full reload of the blockchain db
15:55:06 <hyc> this bug means only 8 of 32byts of hash are being compared, so collisions could be possible
15:57:37 <hyc> dunno the significance of that, dunno what txs_prunable_hash is for
16:00:24 <xnbya> I have a question regarding RPC SSL, does anyone know if is there a reason that the wallet does not use the system ca list?
16:02:37 <hyc> as I recall, the point is not to make nodes identifiable
16:02:52 <hyc> using a public CA would defeat that purpose
16:03:52 <mj-xmr> selsta: I have reproduced the problem by setting the timeout artificially to 1, and I got the same message.
16:04:56 <selsta> mj-xmr: guess our best bet will be to merge it and wait if it shows up again
16:05:18 <mj-xmr> I also observed the CPU load, and indeed, the RX takes all the cores on initialization, just like xmrig miner. If this power is taken up by other processes, it's very natural for the timeout to be too small.
16:14:16 <hyc> marcus40: #7374
16:19:38 <marcus40> yes
16:19:54 <marcus40> that's right yeah....i thought it should be hash32
16:20:31 <hyc> I'm going to dump my current DB and see if the change makes any difference on reimport
16:20:40 <marcus40> do you mind if I PR it
16:20:50 <marcus40> I am trying to contribute to Monero more
16:20:54 <hyc> wdy mean, it is a PR already
16:21:01 <marcus40> lol...
16:21:04 <hyc> and it's not acceptable until we know the effect
16:21:12 <hyc> impact
16:21:36 <sethsimmons> About to publish a new blog post about how to contribute, would love any feedback (specifically on the "filing issues" and "for developers" sections) before I push it: https://paste.debian.net/1185090/
16:21:42 <sethsimmons> Don't want to cause more harm than good 🙂
16:22:04 * hyc reads, ragequits ....    :P
16:22:18 <marcus40> PR already because you swooped in
16:22:22 <marcus40> lol
16:22:47 <marcus40> never mind :')
16:23:03 <hyc> like I said, the PR isn't acceptable until we know what the impact will be
16:23:14 <hyc> if there have been no collisions so far, then we're ok
16:24:00 <marcus40> alright
16:24:03 <hyc> have to check with moneromooo what the significance of this really is
16:24:04 <marcus40> @hyc
16:26:28 <marcus40> when we do mdb_set_dupsort
16:26:40 <marcus40> that is for sorting keys only right?
16:27:28 <hyc> set_compare is for keys
16:27:33 <hyc> set_dupsort is for dup values
16:29:40 <marcus40> on your commit why have you put two entries?
16:29:52 <marcus40> There only needs to be one
16:29:53 <hyc> both are needed
16:30:07 <hyc> hmm.
16:30:20 <hyc> I see, the DB is created with MDB_INTEGERKEY so set_compare is not needed
16:32:28 <hyc> and the set_compare on txs_prunable is also not needed
16:33:04 <marcus40> why does that preclude it from needing a compare?
16:33:20 <hyc> MDB_INTEGERKEY already means compare the key as an integer
16:33:21 <marcus40> just because it's the native format?
16:33:27 <marcus40> ok
16:33:37 <marcus40> got you
17:14:35 <marcus40> hyc if i'm going to store say a std::string consisting of the concatenation of three std::strings , do I need to convert to a C string before putting it in the db for the string compare to work properly?
17:14:53 <marcus40> sorry i meant if i have a string like that for BOTH the key and the value
17:15:07 <hyc> you certainly cannot store std::string objects directly
17:15:37 <marcus40> yeah, but is c_str the way to go?
17:16:12 <hyc> LMDB doesn't do string compares
17:16:44 <hyc> it does set-of-bytes-of-a-given-length compares
17:16:47 <marcus40> BlockchainLMDB::compare_string
17:17:39 <marcus40> ( i know thats what's going on under the hood)
17:18:00 <marcus40> but it's there, so would it make sense to just use that as the compare & dupsort?
17:28:30 <marcus40> also you said that the dupsort's are for values
17:28:43 <marcus40> but we're storing structures of block data here:
17:28:44 <marcus40>   MDB_val_set(val, bi);
17:28:45 <marcus40>   result = mdb_cursor_put(m_cur_block_info, (MDB_val *)&zerokval, &val, MDB_APPENDDUP);
17:28:54 <marcus40> bi is a structure of block data
17:29:17 <marcus40> but we have   mdb_set_dupsort(txn, m_block_info, compare_uint64);
17:38:48 <moneromooo> The data can be anything, but the key is uint64_t.
17:39:26 <moneromooo> That table doesn't need to have duplicate keys I think.
17:39:38 <moneromooo> So dupsort can be, er, whatever hte other one is.
17:39:42 <moneromooo> compare IIRC
17:40:08 <marcus40> but hyc said that dupsort is for keys
17:40:21 <marcus40> sorry, for values
17:40:37 <marcus40> and it takes a compare function as an argument
17:40:59 <moneromooo> Using the system CA list would mean any random CA on the system could sign a cert the wallet would silently accept (at first approximation, it might not do so if we use IPs directly).
17:41:20 <moneromooo> If you want to check the cert, whitelisting it is much safer. Then you know it's the right one, not one some random CA told you is OK.
17:41:37 <moneromooo> But vtnerd added the ability to use system CAs later on.
17:43:06 <moneromooo> MDB_INTEGERKEY bypasses the compare funcs. I didn't know that :/
17:44:01 <moneromooo> dupsort is for values but it implies compare IIRC. hyc ?
17:45:16 <hyc> dupsort only sets the comparator for the values
17:45:25 <hyc> doesn't touch anything for the keys
17:46:25 <marcus40> yeah so that's what I'm saying
17:46:36 <marcus40> block info has values that are blobs of mixed data
17:46:49 <marcus40> but the comparator is uint64_t
17:46:57 <marcus40> what gives
17:47:40 <moneromooo> OK, so it should be setting compare, not dupsort. Do you want to PR that marcus40 ?
17:49:12 <moneromooo> The first element of the block info structure is a uint64_t. The key happens to be empty.
17:50:24 <moneromooo> Setting compare or just remocing the useless dupsort. I don't mind either.
17:51:27 <marcus40> ok, but just checking with you first
17:51:29 <marcus40> MDB_val_set(val_bi, height);
17:51:29 <marcus40>   result = mdb_cursor_get(m_cur_block_info, (MDB_val *)&zerokval, &val_bi, MDB_GET_BOTH);
17:51:30 <marcus40>   if (result == MDB_NOTFOUND)
17:51:30 <marcus40>   {
17:51:31 <marcus40>     throw0(BLOCK_DNE(std::string("Attempt to set cumulative difficulty from height ").append(boost::lexical_cast<std::string>(height)).append(" failed -- difficulty not in db").c_str()));
17:51:31 <marcus40>   }
17:51:32 <marcus40>   else if (result)
17:51:32 <marcus40>     throw0(DB_ERROR("Error attempting to set a cumulative difficulty"));
17:51:33 <marcus40>   mdb_block_info *result_bi = (mdb_block_info *)val_bi.mv_data;
17:51:33 <marcus40>   mdb_block_info bi;
17:51:34 <marcus40>   bi.bi_height = result_bi->bi_height;
17:51:34 <marcus40>   bi.bi_timestamp = result_bi->bi_timestamp;
17:51:35 <marcus40>   bi.bi_coins = result_bi->bi_coins;
17:51:35 <marcus40>   bi.bi_weight = result_bi->bi_weight;
17:51:36 <marcus40>   //bi.bi_diff_lo = diff; // TODO
17:51:36 <marcus40>   bi.bi_hash = result_bi->bi_hash;
17:51:37 <marcus40>   MDB_val_set(val, bi);
17:51:37 <marcus40>   result = mdb_cursor_put(m_cur_block_info, (MDB_val *)&val_bi, &val, MDB_CURRENT);
17:52:06 <marcus40> here it's using the height as the key , and the entire info as the value
17:52:10 <moneromooo> Don't spam here. Put this on a paste ste please.
17:52:19 <marcus40> sorry
17:52:23 <moneromooo> paste.debian.net is nice.
17:52:28 <marcus40> ok i will in future
17:53:24 <marcus40> I guess here MDB_INTEGERKEY is accounting for the integer key
17:53:25 <moneromooo> MDB_CURRENT means it'll add to what was found already.
17:53:56 <marcus40> is it not bad that we dont have a dupsort?
17:53:56 <moneromooo> I assume the size of the value is ignored here.
17:54:14 <moneromooo> This one's always append, it's not sorted.
17:54:26 <moneromooo> Well, it could be I guess since it's sorted by height...
17:56:07 <marcus40> in other places, we only have result = mdb_cursor_put(m_cur_block_info, (MDB_val *)&zerokval, &val, MDB_APPENDDUP);
17:56:24 <marcus40> no key this time?
17:57:57 <moneromooo> A placeholder key looks like.
17:58:21 <moneromooo> Well, real key, but empty.
17:58:46 <moneromooo> So AFAICT it's not used in practice, the value is added at the end.
17:59:12 <moneromooo> I'm getting a feeling I should let hyc address these things as he's the expert there
18:00:40 <marcus40> no it is... if there is no key, constant 'zerokey' concatenated with the 'data' becomes the key to access the data
18:01:10 <marcus40> however what i'm saying is some calls to put (m_cur_block_info
18:01:10 <marcus40> are using a placeholder key and some aren't
18:02:56 <marcus40> is that just because there are multiple types of data being stored in the same kvs?
18:03:08 <moneromooo> In that table, no. Only the structure.
18:09:07 <marcus40> I think I understand what might be happening
18:11:47 <marcus40> But beings as we have no compator for the structure, that is prone to corruption right?
18:12:10 <marcus40> we need a dupsort (tx, ......, compare_struct)?
18:12:25 <marcus40> well compare_block_info
18:40:58 <marcus40> I don't fully understand why yet, but it appears that
18:40:59 <marcus40> (m_cur_block_info, (MDB_val *)&zerokval, &result, MDB_GET_BOTH);
18:40:59 <marcus40> where result is the height
18:41:00 <marcus40> allows you to pull a record created via:
18:41:00 <marcus40> result = mdb_cursor_put(m_cur_block_info, (MDB_val *)&val_bi, &val, MDB_CURRENT);
18:41:01 <marcus40> where val_bi is a height and val is the block data
18:42:03 <marcus40> I guess it just means that (m_cur_block_info, (MDB_val *)&zerokval, &result, MDB_GET_BOTH);  is equivalent to looking up by height
19:08:57 <marcus40> @hyc
19:09:28 <marcus40> if compare() is being used for comparing key in application vs key in database
19:10:09 <marcus40> and dupsort is being used to compare values in application vs values in db,     *when dupsort is enabled*
19:10:22 <marcus40> what about when dupsort isn't enabled? where is our value comparator then
19:10:45 <marcus40> or am I missing the purpose of dupsort
19:16:14 <hyc> everything else in there is correct
19:16:28 <hyc> so whatever you're looking at, you've probably overlooked something
19:17:51 <marcus40> but what is used for value vs value comparison when we *dont* have dupsort enabled
19:18:21 <hyc> please go read the LMDB docs
19:18:24 <marcus40> i did
19:18:28 <marcus40> it doesn't say
19:18:46 <hyc> then use your brain
19:19:05 <hyc> in a key value store, normally a key has only one value
19:19:13 <hyc> since there is only one value, there is no need for a value comparator
19:21:42 <marcus40> but i thought the purpose of the comparator was to check the application value thats searched for, versus what comes out of the db
19:22:18 <hyc> that's not what a key value store does
19:22:35 <hyc> you give it a key to search for, it give you the corresponding value - whatever it is
19:22:56 <hyc> there is no comparison check on values in a normal k/v
19:23:39 <hyc> but LMDB supports multiple sorted values per key, in dupsort tables.
19:24:03 <hyc> ONLY in dupsort tables does it do value comparisons, and need a value comparator. which is what the set_dupsort function provides.
19:24:57 <hyc> this is why the get option is called GET_BOTH - that means check both the key and the value
19:25:08 <hyc> if you don't use GET_BOTH, the value is not checked.
19:25:46 <marcus40> Right I see
19:25:51 <marcus40> thanks very much I get it now
19:26:06 <marcus40> I have one question then pertaining to monero
19:26:45 <marcus40> for the block_info table, we're storing a data structure. why is the comparator a uint64_t
19:26:53 <marcus40> the value comparator
19:29:22 <marcus40> shouldn't there be a custom comparator for the block info structure
19:29:24 <hyc> mooo already answered you. the first field of the struct is a uint64
19:30:00 <hyc> the rest of the fields are irrelevant for matching purposes
19:30:26 <marcus40> but that will laways be the same
19:30:32 <marcus40> the first field is the height of the block
19:31:01 <marcus40> (which is also the key)
19:31:17 <hyc> there is only one block_info per block height so what's the problem?
19:31:42 <hyc> there is only one block for a given height
19:31:44 <marcus40> what about alt blocks that have the same height
19:31:57 <hyc> they aren't in this table
19:31:58 <marcus40> doesn't it store those in there too? orphans
19:32:26 <marcus40> if there is only one block per height it shouldn't be dupsort in the first place then right?
19:32:46 <marcus40> Am I ok to remove the dupsort( ... ..... ., uint64_t compare)
19:33:00 <hyc> wrong and don't touch
19:33:24 <marcus40> so no duplicate values... why dupsort?
19:33:45 <hyc> Read the DB schema description more carefully
19:33:50 <hyc> everything is already explained
19:33:56 <moneromooo> Please try and be a bit less harsh, you're sounding like fireice sometimes :P
19:34:17 <moneromooo> They're all duplicates. That table has a single key (that empty one I mentioned above).
19:34:21 <hyc> this is all in the source already, these are stupid questions.
19:34:31 <hyc> this is all *documented* already
19:34:37 <moneromooo> It takes a while to grok everything.
19:35:33 <hyc> marcus40: read all of this comment https://github.com/monero-project/monero/blob/master/src/blockchain_db/lmdb/db_lmdb.cpp#L189
19:35:37 <moneromooo> And the empty key trick is not something you expect.
19:35:54 <hyc> that's why it is explicitly documented there
19:37:49 <hyc> and it's a cardinal sin to suggest changes to code that you haven't read thoroughly
19:38:03 <moneromooo> What can I say. I'm a sinner...
19:38:27 <marcus40> well i suggested one earlier and it turned out to be right
19:38:48 <moneromooo> Do you mean it was just luck ? :)
19:38:55 <hyc> lol
19:40:25 <marcus40> you quickly lapped that up hyc eh
19:40:28 <marcus40> haha
19:41:26 <marcus40> but fair enough.... new to this part of the code
19:41:53 <marcus40> I feel like the comments are often poor
19:42:02 <marcus40> Not very clear/ thorough
19:43:13 <marcus40> but thanks and see you
19:44:44 <hyc> running an import now with the dupsort patch to see how different the DBs are
19:53:10 <moneromooo> It should not matter, this call was pointless IIRC ?
19:53:40 <moneromooo> That was for a non DUPSORT table, right ?
19:54:03 <hyc> txs_prunable_hash is a dupsort table tho
19:54:14 <hyc> is there only 1 hash per ID?
19:54:19 <hyc> I guess that would be the case
19:55:35 <hyc> ok now I see
19:55:39 <hyc> there's no bug
19:57:39 <mj-xmr> Oh bloody hell with that Copyright.
19:57:57 <mj-xmr> #7334
20:00:34 <hyc> the only thing I'd point out is that the txs_prunable_hash table is created as dupsort but doesn't actually have any duplicate values
20:02:11 <hyc> so no bug, just a little misleading
20:03:26 <moneromooo> Ah. I don't actually rememver how it works now...
20:03:57 <hyc> it's a plain jane key/value table. key is txid, value is tx hash
20:04:32 <moneromooo> OK, then dropping the flag seems like a good idea, yes.
20:04:39 <hyc> it could have been setup as dupfixed to save some space
20:04:51 <moneromooo> Can't recall.
20:04:56 <hyc> but then it would have to do the key prefix trick etc. which it doesn't do
20:05:35 <hyc> at this point we can't really change it. LMDB would complain on dbi_open that the flags are mismatched
20:05:48 <hyc> (when opening an existing DB)
20:05:52 <moneromooo> Oh I see.
20:14:17 <moneromooo> vtnerd: around ?
20:16:44 <vtnerd> yes
20:17:10 <moneromooo> I'm replacing vector with span as you suggested, and I'm getting this:
20:17:47 <moneromooo> https://paste.debian.net/hidden/0a09a22b/
20:18:08 <moneromooo> The ctor I'm aiming for (const T*, size_t) can't be used due to safe_conversion being false.
20:18:22 <moneromooo> But the struct is nothing fancy, just a struct { unsigned char[32]; }
20:18:31 <moneromooo> Do you see why offhand ?
20:20:26 <moneromooo> Line 4800 is now: return bulletproof_PROVE(epee::span<rct::key>(&sv, 1), rct::keyV(1, gamma));
20:20:37 <moneromooo> (the second one will be a span too after this is sorted)
20:23:17 <vtnerd> it looks like const-correctness for at least some of the errors. `epee::span<const rct::key>(&sv, 1)` -> mismatched types `rct::key [N]` and `const rct::key*`"
20:24:09 <vtnerd> well it wouldn't use that overload, but thats the first set of errors
20:24:36 <moneromooo> Oh right. That's one thing that confused me earlier, vector doesn't work like this. Thanks.
20:25:50 <vtnerd> oh I think the other errors are saying it cannot call the other overload
20:29:09 <moneromooo> Indeed, it was just that. I should have worked it out -_- Thanks.
20:33:56 <nojlbbibsm> FIY, you want to know why you are getting those daily messages? Cults NEED to control information. That's why people like lh1008 get savagly attacked (also called disconnection) for even talking to me. Break the information embargo, break the cult. Who would sign up to Scientology if they knew from the get-go that is is about a sci-fi alien overlord? Who would sign up to MAGA if they knew what a loser
20:33:56 <nojlbbibsm> Trump is?
20:33:56 <nojlbbibsm> You don't sign up to MAGA to do a failed Viking LARP and some time off in club fed. You sign up because you feel like shit and they promise to give you self-respect. It is a total lie of course, you are still a joke even when cosplaying a Viking. Just like Monero's promise of privacy is a lie.
20:46:45 <moneromooo> OMFG, fuck this waste of time for just two extra fucking cycles -_-
20:47:20 <moneromooo> Fucking spans. They can go in later.
20:47:28 * moneromooo undoes
20:52:38 <vtnerd> it wasn't an easy transition? and its probably way more than two cycles given the allocation
20:54:54 <moneromooo> Sure, but it's one alloc for the whole of PROVE or VERIFY. De minimis.
20:55:15 <moneromooo> And no, not easy. span is missing a ctor from const U* AFAICT, but adding it did not help either.
20:56:59 <moneromooo> I mean, it might be easy when you're used to C++ery, but it's wasting my time.
20:58:18 <moneromooo> And I'll be back in an hour when the tests have run with the remaining changes.
20:58:21 * moneromooo grumbles
21:29:00 <omega> how to debug this monero-wallet-rpc error? Error calling gettransactions daemon RPC: r 1, status <error>
21:42:03 <vtnerd> why does it need another constructor? none of those types have bases for conversion
21:53:31 <moneromooo> It has a ctor with U * const src_ptr, but none with const U *src_ptr
21:54:17 <moneromooo> So if you have a const object to start with, &object will be const, and so can't bind AFAIK.
21:54:46 <moneromooo> Anyway, I reset and redid the other changes. If you want to switch to span later, you're welcome (also in bulletproofs, which has vectors too).
21:54:59 <moneromooo> But it's just taking me way too much time for not much gain.