Commit Graph

298 Commits

Author SHA1 Message Date
moneromooo-monero 55363c5941 Avoid some temporary strings when reading off the database 2020-08-17 14:02:27 +00:00
Alexander Blair 9871fefbf9 Merge pull request #6565
72cdfa4a2 fix a few typos in error messages (moneromooo-monero)
2020-07-19 03:40:19 -07:00
Alexander Blair 6d6c691a0f Merge pull request #6536
bd9653663 db_lmdb: test for mmap support at init time (moneromooo-monero)
2020-07-19 03:37:15 -07:00
Alexander Blair 36d50d93f2 Merge pull request #6534
7bd66b01b daemon: guard against rare 'difficulty drift' bug with checkpoints and recalculation (stoffu)
2020-07-19 03:36:39 -07:00
stoffu 7bd66b01bf daemon: guard against rare 'difficulty drift' bug with checkpoints and recalculation
On startup, it checks against the difficulty checkpoints, and if any mismatch is found, recalculates all the blocks with wrong difficulties. Additionally, once a week it recalculates difficulties of blocks after the last difficulty checkpoint.
2020-06-09 10:40:51 +09:00
moneromooo-monero 72cdfa4a20 fix a few typos in error messages
Reported by adrelanos
2020-05-19 15:59:26 +00:00
moneromooo-monero bd96536637 db_lmdb: test for mmap support at init time
It'll make it clearer when a DB init failure is due to being
on a filesystem which does not support mmap
2020-05-16 00:20:22 +00:00
SomaticFanatic 5ef0607da6 Update copyright year to 2020
Update copyright year to 2020
2020-05-06 22:36:54 -04:00
luigi1111 7954f8cb18 Merge pull request #6358
8958b4e blockchain_db: faster fetching of consecutive txes (moneromooo-monero)
2020-04-10 16:01:37 -05:00
luigi1111 d102a72027 Merge pull request #6384
ec01077 db_lmdb: fix race crash using a stale cursor (moneromooo-monero)
2020-04-04 13:14:19 -05:00
moneromooo-monero ec01077720 db_lmdb: fix race crash using a stale cursor
If a db resize happened, the txpool meta cursor might be stale,
and was not being renewed when necessary.
It would cause this SEGSEGV:

in mdb_cursor_set ()
in mdb_cursor_get ()
in cryptonote::BlockchainLMDB::get_txpool_tx_blob(crypto::hash const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, cryptonote::relay_category) const ()
in cryptonote::tx_memory_pool::get_transaction(crypto::hash const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, cryptonote::relay_category) const ()
in cryptonote::t_cryptonote_protocol_handler<cryptonote::core>::handle_notify_new_fluffy_block(int, epee::misc_utils::struct_init<cryptonote::NOTIFY_NEW_FLUFFY_BLOCK::request_t>&, cryptonote::cryptonote_connection_context&) ()
2020-03-12 00:57:32 +00:00
moneromooo-monero 8958b4e7aa blockchain_db: faster fetching of consecutive txes
Useful for wallet refresh or node sync
2020-02-27 15:05:34 +00:00
Interchained c61abf87c0 remove empty statements
Cleaning up a little around the code base.
2020-02-17 11:55:15 -05:00
Alexander Blair 8eedc8a390 Merge pull request #6140
08635a08 blockchain: speedup fetching pruned contiguous tx blobs (moneromooo-monero)
2020-01-25 16:55:18 -08:00
Alexander Blair e51ce477db Merge pull request #6157
402dd207 db_lmdb: guard against non NUL terminated keys (moneromooo-monero)
2020-01-24 20:07:37 -08:00
moneromooo-monero 402dd2078c db_lmdb: guard against non NUL terminated keys 2019-11-19 11:12:49 +00:00
moneromooo-monero 08635a0875 blockchain: speedup fetching pruned contiguous tx blobs
About twice as fast, very roughly
2019-11-15 13:23:30 +00:00
Lee Clagett 5d7ae2d279 Adding support for hidden (anonymity) txpool 2019-11-02 20:36:03 +00:00
moneromooo-monero 37cccb8cfb db_lmdb: an empty prunable data record means the tx is not pruned
as opposed to an absent record
2019-10-17 17:39:05 +00:00
moneromooo-monero 8330e772f1 monerod can now sync from pruned blocks
If the peer (whether pruned or not itself) supports sending pruned blocks
to syncing nodes, the pruned version will be sent along with the hash
of the pruned data and the block weight. The original tx hashes can be
reconstructed from the pruned txes and theur prunable data hash. Those
hashes and the block weights are hashes and checked against the set of
precompiled hashes, ensuring the data we received is the original data.
It is currently not possible to use this system when not using the set
of precompiled hashes, since block weights can not otherwise be checked
for validity.

This is off by default for now, and is enabled by --sync-pruned-blocks
2019-09-27 00:10:37 +00:00
moneromooo-monero 4672b5cea9 db_lmdb: print percentages as percentages, not ratios 2019-09-04 14:53:59 +00:00
Howard Chu 50ec40ea16 Increase max_dbs from 20 to 32
We've added a lot of new indices recently, and 20 isn't enough for them plus
new DBs opened during format migrations.
2019-08-18 17:10:45 +01:00
luigi1111 4b76656f5c Merge pull request #5524
06b8f29 blockchain: keep alternative blocks in LMDB (moneromooo-monero)
2019-07-24 14:04:16 -05:00
luigi1111 e3de4aa68b Merge pull request #5502
25a7cfd add a few checks where it seems appropriate (moneromooo-monero)
1a66a86 remove unused code (moneromooo-monero)
2019-07-24 13:57:06 -05:00
moneromooo-monero 2ff99fb47f db_lmdb: commit pruning txn at checkpoints
to avoid errors when the txn is too large
2019-06-15 09:50:54 +00:00
moneromooo-monero 0564da5fdc ensure no NULL is passed to memcpy
NULL is valid when size is 0, but memcpy uses nonnull attributes,
so let's not poke the bear
2019-06-14 08:47:29 +00:00
moneromooo-monero 06b8f29992 blockchain: keep alternative blocks in LMDB
Alternative blocks are cleared on startup unless --keep-alt-blocks
is passed on the command line
2019-05-08 17:36:52 +00:00
moneromooo-monero e980938210 fix wide difficulty conversion with some versions of boost 2019-05-01 19:58:09 +00:00
moneromooo-monero 25a7cfdb4a add a few checks where it seems appropriate 2019-04-29 20:52:40 +00:00
moneromooo-monero 5e673c03fe blockchain_db: fix db txn ending too early
The db txn in add_block ending caused the entire overarching
batch txn to stop.
Also add a new guard class so a db txn can be stopped in the
face of exceptions.

Also use a read only db txn in init when the db itself is
read only, and do not save the max tx size in that case.
2019-04-14 08:35:38 +00:00
moneromooo-monero 089c7637a6 cryptonote: rework block blob size sanity check
Use the actual block weight limit, assuming that weight is always
greater or equal to size
2019-04-05 09:35:19 +00:00
moneromooo-monero 91f4c7f45f Make difficulty 128 bit instead of 64 bit
Based on Boolberry work by:
  jahrsg <jahr@jahr.me>
  cr.zoidberg <crypto.zoidberg@gmail.com>
2019-03-24 21:03:19 +00:00
Riccardo Spagni 5ac46c5310 Merge pull request #5256
4b21d38d blockchain: speed up getting N blocks weights/long term weights (moneromooo-monero)
2019-03-21 14:48:40 +02:00
moneromooo-monero 4b21d38dfd blockchain: speed up getting N blocks weights/long term weights 2019-03-08 12:04:14 +00:00
binaryFate 1f2930ce0b Update 2019 copyright 2019-03-05 22:05:34 +01:00
Riccardo Spagni e65221d690 Merge pull request #5119
f024a10b db_lmdb: make mdb_block_info format conversion more future proof (moneromooo-monero)
2019-03-05 14:12:24 +02:00
moneromooo-monero f024a10b68 db_lmdb: make mdb_block_info format conversion more future proof
If mdb_block_info changes again, the v2 to v3 conversion would
convert to an incorrect format.
2019-03-05 12:06:23 +00:00
moneromooo-monero 79b4e9f377 save some database calls when getting top block hash and height 2019-03-05 11:58:05 +00:00
moneromooo-monero b044d03a51 Avoid repeated (de)serialization when syncing 2019-03-05 11:57:55 +00:00
moneromooo-monero e69477bf25 db: speedup block addition
by avoiding repeated (de)serialization
2019-03-05 10:28:52 +00:00
Riccardo Spagni 11daa98c0a Merge pull request #5081
d78addcb db_lmdb: don't unnecessarily cast to double on the way to uint64_t (moneromooo-monero)
2019-03-04 21:18:24 +02:00
Riccardo Spagni 09020235bf Merge pull request #5175
85088d9f db_lmdb: fix missing mdb_dbi_close in migration (moneromooo-monero)
2019-03-04 17:06:58 +02:00
moneromooo-monero b8787f4302 ArticMine's new block weight algorithm
This curbs runaway growth while still allowing substantial
spikes in block weight

Original specification from ArticMine:

here is the scaling proposal
Define: LongTermBlockWeight
Before fork:
LongTermBlockWeight = BlockWeight
At or after fork:
LongTermBlockWeight = min(BlockWeight, 1.4*LongTermEffectiveMedianBlockWeight)
Note: To avoid possible consensus issues over rounding the LongTermBlockWeight for a given block should be calculated to the nearest byte, and stored as a integer in the block itself. The stored LongTermBlockWeight is then used for future calculations of the LongTermEffectiveMedianBlockWeight and not recalculated each time.
Define:   LongTermEffectiveMedianBlockWeight
LongTermEffectiveMedianBlockWeight = max(300000, MedianOverPrevious100000Blocks(LongTermBlockWeight))
Change Definition of EffectiveMedianBlockWeight
From (current definition)
EffectiveMedianBlockWeight  = max(300000, MedianOverPrevious100Blocks(BlockWeight))
To (proposed definition)
EffectiveMedianBlockWeight  = min(max(300000, MedianOverPrevious100Blocks(BlockWeight)), 50*LongTermEffectiveMedianBlockWeight)
Notes:
1) There are no other changes to the existing penalty formula, median calculation, fees etc.
2) There is the requirement to store the LongTermBlockWeight of a block unencrypted in the block itself. This  is to avoid possible consensus issues over rounding and also to prevent the calculations from becoming unwieldy as we move away from the fork.
3) When the  EffectiveMedianBlockWeight cap is reached it is still possible to mine blocks up to 2x the EffectiveMedianBlockWeight by paying the corresponding penalty.

Note: the long term block weight is stored in the database, but not in the actual block itself,
since it requires recalculating anyway for verification.
2019-03-04 09:33:58 +00:00
moneromooo-monero 85088d9f85 db_lmdb: fix missing mdb_dbi_close in migration
Fixed by hyc
2019-02-21 09:52:30 +00:00
moneromooo-monero b750fb27b0 Pruning
The blockchain prunes seven eighths of prunable tx data.
This saves about two thirds of the blockchain size, while
keeping the node useful as a sync source for an eighth
of the blockchain.

No other data is currently pruned.

There are three ways to prune a blockchain:

- run monerod with --prune-blockchain
- run "prune_blockchain" in the monerod console
- run the monero-blockchain-prune utility

The first two will prune in place. Due to how LMDB works, this
will not reduce the blockchain size on disk. Instead, it will
mark parts of the file as free, so that future data will use
that free space, causing the file to not grow until free space
grows scarce.

The third way will create a second database, a pruned copy of
the original one. Since this is a new file, this one will be
smaller than the original one.

Once the database is pruned, it will stay pruned as it syncs.
That is, there is no need to use --prune-blockchain again, etc.
2019-01-22 20:30:51 +00:00
moneromooo-monero d78addcbc0 db_lmdb: don't unnecessarily cast to double on the way to uint64_t 2019-01-19 00:03:04 +00:00
moneromooo-monero a6216d1ac2 blockchain_db: allow getting output keys without commitment
Since the commitment has to be calculated for non rct outputs,
it slows down a lot unnecessarily if we don't need it
2019-01-16 20:13:08 +00:00
Riccardo Spagni 9d57ee9ff6 Merge pull request #4984
008647d7 blockchain_db: speedup tx output gathering (moneromooo-monero)
2019-01-16 19:06:07 +02:00
moneroexamples ec68cf822f Make get_output_key method const
get_output_key method is commonly used when working with txs and their key images. Because the method is not const, passing blockchain object though const& or pointers to const is not possible in this context. This is especially problematic in external projects (e.g., projects in moneroexamples) that use monero C++ api to operate on the blockchain and txs.

Thus, having get_output_key method will simplify moving blockchain object around through const references and pointers to const objects.
2019-01-07 05:53:14 +08:00
luigi1111 69e8567c0e Merge pull request #4946
6644b9b blockchain_db: remove a couple unused functions (moneromooo-monero)
ce594f5 blockchain_db: allocate known size vector only once (moneromooo-monero)
8332698 db_lmdb: inline check_open, it's trivial and called everywhere (moneromooo-monero)
5511563 db_lmdb: avoid pointless division (moneromooo-monero)
d1efe3d cryptonote: set tx hash on newly parsed txes when known (moneromooo-monero)
9cc68a2 tx_pool: add a few std::move where it can make a difference (moneromooo-monero)
2018-12-31 16:06:50 -06:00