Commit Graph

122 Commits

Author SHA1 Message Date
Howard Chu ee7a8b87f9 Get rid of lmdb_cur
We don't need it now with per-txn cursors.
2016-03-03 19:06:27 +00:00
warptangent 2c823c1589 Make partial revert consistent
If user-defined comparator is used, subdb shouldn't be opened with
MDB_INTEGERKEY.

TODO: Again, this will be added back with future schema updates.
2016-03-01 02:47:16 -08:00
warptangent df239428c0 Revert part of 7db89ed2ee
For now, so existing databases work.

TODO: add these back with future schema updates.
2016-02-29 18:03:41 -08:00
warptangent c9c4060dc6 BlockchainLMDB: Add height and db error to exception 2016-02-29 17:52:54 -08:00
Riccardo Spagni d60bf4ee36 Merge pull request #686
7db89ed ARMv7: fix unaligned accesses (Howard Chu)
5a07cef Wrap some more actions in a larger read txn (Howard Chu)
8cc7a36 read txn/cursor stuff (Howard Chu)
86a7f2b core: check whether an update is needed straight away (moneromooo-monero)
ea5fa5e core: print "update needed" hard fork notifications in red (moneromooo-monero)
2016-02-29 17:29:26 -05:00
Howard Chu 7db89ed2ee ARMv7: fix unaligned accesses
And cleanup some key comparators
2016-02-25 13:57:00 +00:00
Howard Chu 8cc7a36f0b read txn/cursor stuff
Could wrap more later.
2016-02-23 20:47:15 +00:00
moneromooo-monero 0eb4d054f6 db_bdb: add missing include for std::condition_variable 2016-02-20 15:38:43 +00:00
warptangent 57e75fa33f BlockchainBDB: Check if hard fork subdbs need reset
See f7e337e625 for LMDB equivalent.
2016-02-18 06:38:56 -08:00
warptangent 47f6cf8bea BlockchainBDB: Support blockchain_import --drop-hard-fork command
See c657e772c4 for LMDB equivalent.
2016-02-18 06:37:45 -08:00
Howard Chu 02abe3590d Use MDB_PREV_MULTIPLE
in get_global_output_indices
2016-02-17 16:14:50 +00:00
Howard Chu 7a4755d3a4 Fixup after lmdb master resync 2016-02-17 16:14:38 +00:00
Howard Chu 3b13a7473e Shutup about VERSION 0 2016-02-17 05:06:19 +00:00
Howard Chu 1537477c9f Use cursor in get_output_key 2016-02-17 04:05:29 +00:00
Howard Chu f2faf8cdd9 Use MDB_APPEND mode where possible
When keys are contiguous and monotonically increasing, this gets
denser page utilization (doesn't leave padding in page splits).
Can't be used for keys that are inserted in random order (e.g. hashes)
In total this only saves around 1.5% of space compared to original
DB code. The previous patch accounted for 0.8% savings on its own;
the blocks tables just aren't that big.
2016-02-17 04:05:29 +00:00
Howard Chu 090b548c3b Use cursors in write txns
Saves a bit of seek overhead. LMDB frees them automatically
in txn_(commit|abort) so they need no cleanup.
2016-02-17 04:05:29 +00:00
Howard Chu ed08d2152e Keep a running blocksize count
Used in batch size estimation, avoids rereading already processed
blocks during import
2016-02-17 04:05:28 +00:00
Howard Chu 0fc93345ea Win32 import batchsize tweaks
Reduce frequency of resizes: bump minimum increase from 128MB to 512MB
Use a bigger safety margin at small batch sizes
2016-02-17 04:05:28 +00:00
Riccardo Spagni 63e2d4195b Merge pull request #657
1995923 BlockchainLMDB: Deal with DB exceptions at block level with particularity (warptangent)
c16cc20 BlockchainLMDB: Add sanity check for inconsistent state (warptangent)
9118d0a BlockchainLMDB: Call destructor on allocated txn if setup fails (warptangent)
f5581c3 BlockchainLMDB: Replace remaining txn pointer NULLs with nullptr (warptangent)
2016-02-13 19:54:47 +02:00
warptangent 1995923559 BlockchainLMDB: Deal with DB exceptions at block level with particularity
Add another DB error exception type to distinguish failed txn setup from
general use of txn.

This keeps the error handling flow the same as before the block-level
txn setup changes that moved control up a layer to BlockchainDB.
2016-02-13 05:12:39 -08:00
warptangent c16cc204d3 BlockchainLMDB: Add sanity check for inconsistent state
This hasn't been known to occur in block-level txn abort, but throw
exception if it does.
2016-02-13 04:17:37 -08:00
warptangent 9118d0a44c BlockchainLMDB: Call destructor on allocated txn if setup fails 2016-02-13 04:17:37 -08:00
warptangent f5581c3536 BlockchainLMDB: Replace remaining txn pointer NULLs with nullptr
For consistency.
2016-02-13 04:17:36 -08:00
moneromooo-monero 8649b9f1ef blockchain_db: pass hard fork object as a simple pointer 2016-02-08 20:56:31 +00:00
warptangent e02577f594 Move HardFork DB update to BlockchainDB::add_block()
Ensures the database is consistent.

Also simplifes blockchain_import in that verify mode off has less to
work around.
2016-02-08 09:28:19 -08:00
warptangent 3800875406 Make HardFork object available to BlockchainDB and derived DB implementations
This will later allow the HardFork object's DB update functions to be
called when the DB transaction that persists across block add/remove is
open.
2016-02-08 09:28:17 -08:00
warptangent fd46c96dce BlockchainDB/LMDB: Refactor block-scope DB txn handling for add block
Move block-scope txn start and stop from BlockchainLMDB to BlockchainDB.
2016-02-08 09:28:15 -08:00
warptangent f3a6000094 BlockchainDB/LMDB/BDB: Extract DB txn functions for block add/remove 2016-02-08 09:28:14 -08:00
warptangent f47d5b0fe3 BlockchainLMDB: Allow two HardFork functions to update DB during block add
Note that this doesn't yet cause them to be called during block add.
2016-02-08 09:28:12 -08:00
warptangent 15ee0bef4b BlockchainLMDB: extract txn macros used during block add/remove 2016-02-08 09:28:10 -08:00
warptangent f7e337e625 fake_core: Check if hard fork subdbs need reset at start 2016-02-08 08:50:48 -08:00
warptangent c657e772c4 blockchain_import: Add --drop-hard-fork command 2016-02-08 08:50:47 -08:00
moneromooo-monero 3a0f4d8a57 berkeleydb: fix delete/free mismatch
Despite being C++, the stats object is allocated by the underlying
C layer using malloc(3).
2016-02-03 08:49:11 +00:00
warptangent b8cc6d46bd BlockchainBDB: Continue when tx has no outputs
See 24b66243c2
2016-01-28 21:15:46 -08:00
warptangent b8707466e2 BlockchainBDB: When removing, find amount output index fast by starting at end
This improves blockchain reorganization time by allowing one of the more
expensive DB lookups when popping a block to not have to seek through a
long dup list in the "output_amounts" db. This is most noticeable for
HDDs.

See ffcf6bdb95
2016-01-28 21:15:44 -08:00
warptangent 1a5c3fa729 BlockchainBDB: Remove tx outputs in reverse order
Data should be removed in the reverse order it was added.

This matches the order of removal in
blockchain_storage::pop_transaction_from_global_index.

See f11def012f
2016-01-28 21:15:42 -08:00
me0wmix 28f95eb001 OpenBSD support for Monero. 2016-01-21 11:18:26 -07:00
warptangent 24b66243c2 db_lmdb: Continue when tx has no outputs 2016-01-19 16:55:31 -08:00
Howard Chu 30f92f5630 Fix hf when import with verify off
Delete the hf tables, so the next open will rescan and regenerate
2016-01-15 17:26:19 +00:00
Riccardo Spagni de03926850 updated copyright year 2015-12-31 08:39:56 +02:00
Riccardo Spagni 810a11267c fixed copyrights with bad year references 2015-12-31 08:37:27 +02:00
Riccardo Spagni fd36eea6dd Merge pull request #572
b39aae7 Tweak 45800a25e9 (hyc)
4a5a5ff blockchain: always stop the ioservice before returning (moneromooo-monero)
78b65cf db_lmdb: safety close db at exit (moneromooo-monero)
45800a2 db_lmdb: fix a strdup/delete[] mistmatch (moneromooo-monero)
2015-12-30 09:38:41 +02:00
hyc b39aae7aa5 Tweak 45800a25e9
trivial cleanup
2015-12-29 00:11:35 +00:00
moneromooo-monero 78b65cf7e8 db_lmdb: safety close db at exit
Noticed by hyc

Also set m_open to false when closed
2015-12-28 19:23:02 +00:00
moneromooo-monero 45800a25e9 db_lmdb: fix a strdup/delete[] mistmatch 2015-12-28 19:22:37 +00:00
warptangent ee9d71e9f9 BlockchainDB: skip fixup check if read-only database 2015-12-26 14:30:20 -08:00
Riccardo Spagni 95ceb715dc Merge pull request #565
79beed2 tests: fix various tests by using parameters better suited to monero (moneromooo-monero)
d0a8362 tests: fix some double spending tests (moneromooo-monero)
2358d0d tests: use 255 as a "too high" block version (moneromooo-monero)
f33a88c blockchain: fix a few block addition bugs (moneromooo-monero)
a9ff11c blockchain: fix an off by one error in unlocked time check (moneromooo-monero)
f294be3 blockchain: reinstate double spending checks in check_tx_inputs (moneromooo-monero)
737b6d6 blockchain: make some flag twiddling code closer to the original (moneromooo-monero)
81cb0fc blockchain: fix bitflipping test with quantized block rewards (moneromooo-monero)
22ddf09 blockchain: add missing m_tx_pool.on_blockchain_dec (moneromooo-monero)
d837c0c blockchain: fix switch to alternative blockchain for more than one block (moneromooo-monero)
5cec076 blockchain: add a missing validity check to rollback_blockchain_switching (moneromooo-monero)
3cabdb5 core: catch exceptions from get_output_key (moneromooo-monero)
5eef645 db: throw when given a non txout_to_key output to add (moneromooo-monero)
2015-12-26 10:23:17 +02:00
moneromooo-monero 5eef64578b db: throw when given a non txout_to_key output to add
The check was explicit in the original version, so it seems
safer to make it explicit here, especially as it is now done
implicitely in a different place, away from the original check.
2015-12-25 21:56:37 +00:00
warptangent d4c2fae2fb BlockchainDB: Remove txs in reverse order
Data should be removed in the reverse order it was added. Not doing so
breaks assumptions and can cause problems in other DB implementations.

This matches the order of tx removal in
blockchain_storage::purge_block_data_from_blockchain.
2015-12-24 15:09:09 -08:00
warptangent ffcf6bdb95 BlockchainLMDB: When removing, find amount output index fast by starting at end
This improves blockchain reorganization time by allowing one of the more
expensive DB lookups when popping a block to not have to seek through a
long dup list in the "output_amounts" subdb. This is most noticeable for
HDDs.

As before, the dup list is still walked if necessary (but in reverse),
and the global output index still confirmed to be the one looked for.
But under proper use, the result will be found at the end of the dup
list, so we start there.

Removing an amount output index is always done in the context of popping
a block, so the global output index being looked for should be the last
one in that amount key's dup list. Even if the txs themselves aren't
removed in reverse order (supposed to be according to original
implementation), the specified amount output index will still be near
the end, because the txs are in the same block.

TEST:

Pop blocks with blockchain_import.

Blocks should be successfully removed with no errors shown.

bitmonerod should be able to start syncing from the reduced blockchain
height.
2015-12-24 13:32:35 -08:00