Merge pull request #8801

1328048 wallet2: fix infinite loop in fake out selection (Crypto City)
wallet2: fix infinite loop in fake out selection
2023-03-27 09:17:55 -04:00 · 2023-03-25 19:26:43 +00:00 · 2023-03-24 22:55:49 -04:00 · 2023-03-24 22:55:29 -04:00 · 2023-03-19 02:58:38 +01:00 · 2023-03-19 02:58:38 +01:00
30 changed files with 795 additions and 145 deletions
@@ -27,10 +27,10 @@ jobs:
    env:
      CCACHE_TEMPDIR: /tmp/.ccache-temp
    steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v3
      with:
        submodules: recursive
-    - uses: actions/cache@v2
+    - uses: actions/cache@v3
      with:
        path: /Users/runner/Library/Caches/ccache
        key: ccache-${{ runner.os }}-build-${{ github.sha }}
@@ -51,15 +51,15 @@ jobs:
      run:
        shell: msys2 {0}
    steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v3
      with:
        submodules: recursive
-    - uses: actions/cache@v2
+    - uses: actions/cache@v3
      with:
        path: C:\Users\runneradmin\.ccache
        key: ccache-${{ runner.os }}-build-${{ github.sha }}
        restore-keys: ccache-${{ runner.os }}-build-
-    - uses: eine/setup-msys2@v2
+    - uses: msys2/setup-msys2@v2
      with:
        update: true
        install: mingw-w64-x86_64-toolchain make mingw-w64-x86_64-cmake mingw-w64-x86_64-ccache mingw-w64-x86_64-boost mingw-w64-x86_64-openssl mingw-w64-x86_64-zeromq mingw-w64-x86_64-libsodium mingw-w64-x86_64-hidapi mingw-w64-x86_64-protobuf-c mingw-w64-x86_64-libusb mingw-w64-x86_64-unbound git
@@ -79,10 +79,10 @@ jobs:
      matrix:
        os: [ubuntu-22.04, ubuntu-20.04]
    steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v3
      with:
        submodules: recursive
-    - uses: actions/cache@v2
+    - uses: actions/cache@v3
      with:
        path: ~/.ccache
        key: ccache-${{ runner.os }}-build-${{ matrix.os }}-${{ github.sha }}
@@ -105,10 +105,10 @@ jobs:
    env:
      CCACHE_TEMPDIR: /tmp/.ccache-temp
    steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v3
      with:
        submodules: recursive
-    - uses: actions/cache@v2
+    - uses: actions/cache@v3
      with:
        path: ~/.ccache
        key: ccache-${{ runner.os }}-libwallet-${{ github.sha }}
@@ -133,11 +133,11 @@ jobs:
    env:
      CCACHE_TEMPDIR: /tmp/.ccache-temp
    steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v3
      with:
        submodules: recursive
    - name: ccache
-      uses: actions/cache@v2
+      uses: actions/cache@v3
      with:
        path: ~/.ccache
        key: ccache-${{ runner.os }}-build-ubuntu-latest-${{ github.sha }}
@@ -167,8 +167,9 @@ jobs:
  source-archive:
    runs-on: ubuntu-20.04
    steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v3
      with:
+        fetch-depth: 0
        submodules: recursive
    - name: archive
      run: |
@@ -177,7 +178,7 @@ jobs:
        export OUTPUT="$VERSION.tar"
        echo "OUTPUT=$OUTPUT" >> $GITHUB_ENV
        /home/runner/.local/bin/git-archive-all --prefix "$VERSION/" --force-submodules "$OUTPUT"
-    - uses: actions/upload-artifact@v2
+    - uses: actions/upload-artifact@v3
      with:
        name: ${{ env.OUTPUT }}
        path: /home/runner/work/monero/monero/${{ env.OUTPUT }}
@@ -57,19 +57,20 @@ jobs:
            packages: "clang-8 gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
    name: ${{ matrix.toolchain.name }}
    steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v3
      with:
+        fetch-depth: 0
        submodules: recursive
 # Most volatile cache
    - name: ccache
-      uses: actions/cache@v2
+      uses: actions/cache@v3
      with:
        path: ~/.ccache
        key: ccache-${{ matrix.toolchain.host }}-${{ github.sha }}
        restore-keys: ccache-${{ matrix.toolchain.host }}-
 # Less volatile cache
    - name: depends cache
-      uses: actions/cache@v2
+      uses: actions/cache@v3
      with:
        path: contrib/depends/built
        key: depends-${{ matrix.toolchain.host }}-${{ hashFiles('contrib/depends/packages/*') }}
@@ -78,7 +79,7 @@ jobs:
          depends-${{ matrix.toolchain.host }}-
 # Static cache
    - name: OSX SDK cache
-      uses: actions/cache@v2
+      uses: actions/cache@v3
      with:
        path: contrib/depends/sdk-sources
        key: sdk-${{ matrix.toolchain.host }}-${{ matrix.toolchain.osx_sdk }}
@@ -96,7 +97,7 @@ jobs:
      run: |
        ${{env.CCACHE_SETTINGS}}
        make depends target=${{ matrix.toolchain.host }} -j2
-    - uses: actions/upload-artifact@v2
+    - uses: actions/upload-artifact@v3
      if: ${{ matrix.toolchain.host == 'x86_64-w64-mingw32' || matrix.toolchain.host == 'x86_64-apple-darwin11' || matrix.toolchain.host == 'x86_64-unknown-linux-gnu' }}
      with:
        name: ${{ matrix.toolchain.name }}
@@ -42,7 +42,7 @@ jobs:
        echo \`\`\` >> $GITHUB_STEP_SUMMARY
        shasum -a256 * >> $GITHUB_STEP_SUMMARY
        echo \`\`\` >> $GITHUB_STEP_SUMMARY
-    - uses: actions/upload-artifact@v3.1.0
+    - uses: actions/upload-artifact@v3
      with:
        name: ${{ matrix.operating-system.name }}
        path: |
@@ -138,8 +138,8 @@ Dates are provided in the format YYYY-MM-DD.
 | 1978433                        | 2019-11-30 | v12               | v0.15.0.0              | v0.16.0.0                  | New PoW based on RandomX, only allow >= 2 outputs, change to the block median used to calculate penalty, v1 coinbases are forbidden, rct sigs in coinbase forbidden, 10 block lock time for incoming outputs
 | 2210000                        | 2020-10-17 | v13               | v0.17.0.0              | v0.17.3.2                  | New CLSAG transaction format
 | 2210720                        | 2020-10-18 | v14               | v0.17.1.1              | v0.17.3.2                  | forbid old MLSAG transaction format
-| 2688888                        | 2022-08-13 | v15               | v0.18.0.0              | v0.18.2.0                  | ringsize = 16, bulletproofs+, view tags, adjusted dynamic block weight algorithm
-| 2689608                        | 2022-08-14 | v16               | v0.18.0.0              | v0.18.2.0                  | forbid old v14 transaction format
+| 2688888                        | 2022-08-13 | v15               | v0.18.0.0              | v0.18.2.1                  | ringsize = 16, bulletproofs+, view tags, adjusted dynamic block weight algorithm
+| 2689608                        | 2022-08-14 | v16               | v0.18.0.0              | v0.18.2.1                  | forbid old v14 transaction format
 | XXXXXXX                        | XXX-XX-XX | XXX                | vX.XX.X.X              | vX.XX.X.X                  | XXX |

 X's indicate that these details have not been determined as of commit date.
@@ -344,7 +344,7 @@ Tested on a Raspberry Pi Zero with a clean install of minimal Raspbian Stretch (
    ```bash
    git clone https://github.com/monero-project/monero.git
    cd monero
-    git checkout v0.18.2.0
+    git checkout v0.18.2.1
    ```

 * Build:
@@ -463,10 +463,10 @@ application.
    cd monero
    ```

-* If you would like a specific [version/tag](https://github.com/monero-project/monero/tags), do a git checkout for that version. eg. 'v0.18.2.0'. If you don't care about the version and just want binaries from master, skip this step:
+* If you would like a specific [version/tag](https://github.com/monero-project/monero/tags), do a git checkout for that version. eg. 'v0.18.2.1'. If you don't care about the version and just want binaries from master, skip this step:

    ```bash
-    git checkout v0.18.2.0
+    git checkout v0.18.2.1
    ```

 * If you are on a 64-bit system, run:
@@ -57,7 +57,7 @@ The dockrun.sh script will do everything to build the binaries. Just specify the
 version to build as its only argument, e.g.

 ```bash
-VERSION=v0.18.2.0
+VERSION=v0.18.2.1
 ./dockrun.sh $VERSION
 ```

@@ -133,7 +133,7 @@ Common setup part:
 su - gitianuser

 GH_USER=YOUR_GITHUB_USER_NAME
-VERSION=v0.18.2.0
+VERSION=v0.18.2.1
 ```

 Where `GH_USER` is your GitHub user name and `VERSION` is the version tag you want to build. 
@@ -244,6 +244,7 @@ namespace cryptonote
    ADD_CHECKPOINT2(2706000, "d8eb144c5e1fe6b329ecc900ec95e7792fccff84175fb23a25ed59d7299a511c", "0x310f7d89372f705");
    ADD_CHECKPOINT2(2720000, "b19fb41dff15bd1016afbee9f8469f05aab715c9e5d1b974466a11fd58ecbb86", "0x3216b5851ddbb61");
    ADD_CHECKPOINT2(2817000, "39726d19ccaac01d150bec827b877ffae710b516bd633503662036ef4422e577", "0x3900669561954c1");
+    ADD_CHECKPOINT2(2844000, "28fc7b446dfef5b469f5778eb72ddf32a307a5f5a9823d1c394e772349e05d40", "0x3af384ec0e97d12");
    return true;
  }

@@ -53,6 +53,7 @@ namespace cryptonote
    bool m_overspend;
    bool m_fee_too_low;
    bool m_too_few_outputs;
+    bool m_tx_extra_too_big;
  };

  struct block_verification_context
@@ -206,6 +206,11 @@

 #define DNS_BLOCKLIST_LIFETIME (86400 * 8)

+//The limit is enough for the mandatory transaction content with 16 outputs (547 bytes),
+//a custom tag (1 byte) and up to 32 bytes of custom data for each recipient.
+// (1+32) + (1+1+16*32) + (1+16*32) = 1060
+#define MAX_TX_EXTRA_SIZE                       1060
+
 // New constants are intended to go here
 namespace config
 {
@@ -248,6 +253,7 @@ namespace config
  const unsigned char HASH_KEY_MM_SLOT = 'm';
  const constexpr char HASH_KEY_MULTISIG_TX_PRIVKEYS_SEED[] = "multisig_tx_privkeys_seed";
  const constexpr char HASH_KEY_MULTISIG_TX_PRIVKEYS[] = "multisig_tx_privkeys";
+  const constexpr char HASH_KEY_TXHASH_AND_MIXRING[] = "txhash_and_mixring";

  // Multisig
  const uint32_t MULTISIG_MAX_SIGNERS{16};
@@ -31,7 +31,9 @@ set(cryptonote_core_sources
  cryptonote_core.cpp
  tx_pool.cpp
  tx_sanity_check.cpp
-  cryptonote_tx_utils.cpp)
+  cryptonote_tx_utils.cpp
+  tx_verification_utils.cpp
+)

 set(cryptonote_core_headers)

@@ -57,6 +57,7 @@
 #include "common/notify.h"
 #include "common/varint.h"
 #include "common/pruning.h"
+#include "common/data_cache.h"
 #include "time_helper.h"

 #undef MONERO_DEFAULT_LOG_CATEGORY
@@ -98,7 +99,8 @@ Blockchain::Blockchain(tx_memory_pool& tx_pool) :
  m_difficulty_for_next_block(1),
  m_btc_valid(false),
  m_batch_success(true),
-  m_prepare_height(0)
+  m_prepare_height(0),
+  m_rct_ver_cache()
 {
  LOG_PRINT_L3("Blockchain::" << __func__);
 }
@@ -3211,7 +3213,7 @@ bool Blockchain::have_tx_keyimges_as_spent(const transaction &tx) const
  }
  return false;
 }
-bool Blockchain::expand_transaction_2(transaction &tx, const crypto::hash &tx_prefix_hash, const std::vector<std::vector<rct::ctkey>> &pubkeys) const
+bool Blockchain::expand_transaction_2(transaction &tx, const crypto::hash &tx_prefix_hash, const std::vector<std::vector<rct::ctkey>> &pubkeys)
 {
  PERF_TIMER(expand_transaction_2);
  CHECK_AND_ASSERT_MES(tx.version == 2, false, "Transaction version is not 2");
@@ -3534,6 +3536,13 @@ bool Blockchain::check_tx_inputs(transaction& tx, tx_verification_context &tvc,
        false, "Transaction spends at least one output which is too young");
  }

+  // Warn that new RCT types are present, and thus the cache is not being used effectively
+  static constexpr const std::uint8_t RCT_CACHE_TYPE = rct::RCTTypeBulletproofPlus;
+  if (tx.rct_signatures.type > RCT_CACHE_TYPE)
+  {
+    MWARNING("RCT cache is not caching new verification results. Please update RCT_CACHE_TYPE!");
+  }
+
  if (tx.version == 1)
  {
    if (threads > 1)
@@ -3555,12 +3564,6 @@ bool Blockchain::check_tx_inputs(transaction& tx, tx_verification_context &tvc,
  }
  else
  {
-    if (!expand_transaction_2(tx, tx_prefix_hash, pubkeys))
-    {
-      MERROR_VER("Failed to expand rct signatures!");
-      return false;
-    }
-
    // from version 2, check ringct signatures
    // obviously, the original and simple rct APIs use a mixRing that's indexes
    // in opposite orders, because it'd be too simple otherwise...
@@ -3578,61 +3581,7 @@ bool Blockchain::check_tx_inputs(transaction& tx, tx_verification_context &tvc,
    case rct::RCTTypeCLSAG:
    case rct::RCTTypeBulletproofPlus:
    {
-      // check all this, either reconstructed (so should really pass), or not
-      {
-        if (pubkeys.size() != rv.mixRing.size())
-        {
-          MERROR_VER("Failed to check ringct signatures: mismatched pubkeys/mixRing size");
-          return false;
-        }
-        for (size_t i = 0; i < pubkeys.size(); ++i)
-        {
-          if (pubkeys[i].size() != rv.mixRing[i].size())
-          {
-            MERROR_VER("Failed to check ringct signatures: mismatched pubkeys/mixRing size");
-            return false;
-          }
-        }
-
-        for (size_t n = 0; n < pubkeys.size(); ++n)
-        {
-          for (size_t m = 0; m < pubkeys[n].size(); ++m)
-          {
-            if (pubkeys[n][m].dest != rct::rct2pk(rv.mixRing[n][m].dest))
-            {
-              MERROR_VER("Failed to check ringct signatures: mismatched pubkey at vin " << n << ", index " << m);
-              return false;
-            }
-            if (pubkeys[n][m].mask != rct::rct2pk(rv.mixRing[n][m].mask))
-            {
-              MERROR_VER("Failed to check ringct signatures: mismatched commitment at vin " << n << ", index " << m);
-              return false;
-            }
-          }
-        }
-      }
-
-      const size_t n_sigs = rct::is_rct_clsag(rv.type) ? rv.p.CLSAGs.size() : rv.p.MGs.size();
-      if (n_sigs != tx.vin.size())
-      {
-        MERROR_VER("Failed to check ringct signatures: mismatched MGs/vin sizes");
-        return false;
-      }
-      for (size_t n = 0; n < tx.vin.size(); ++n)
-      {
-        bool error;
-        if (rct::is_rct_clsag(rv.type))
-          error = memcmp(&boost::get<txin_to_key>(tx.vin[n]).k_image, &rv.p.CLSAGs[n].I, 32);
-        else
-          error = rv.p.MGs[n].II.empty() || memcmp(&boost::get<txin_to_key>(tx.vin[n]).k_image, &rv.p.MGs[n].II[0], 32);
-        if (error)
-        {
-          MERROR_VER("Failed to check ringct signatures: mismatched key image");
-          return false;
-        }
-      }
-
-      if (!rct::verRctNonSemanticsSimpleCached(rv))
+      if (!ver_rct_non_semantics_simple_cached(tx, pubkeys, m_rct_ver_cache, RCT_CACHE_TYPE))
      {
        MERROR_VER("Failed to check ringct signatures!");
        return false;
@@ -3641,6 +3590,12 @@ bool Blockchain::check_tx_inputs(transaction& tx, tx_verification_context &tvc,
    }
    case rct::RCTTypeFull:
    {
+      if (!expand_transaction_2(tx, tx_prefix_hash, pubkeys))
+      {
+        MERROR_VER("Failed to expand rct signatures!");
+        return false;
+      }
+
      // check all this, either reconstructed (so should really pass), or not
      {
        bool size_matches = true;
@@ -5627,7 +5582,7 @@ void Blockchain::cancel()
 }

 #if defined(PER_BLOCK_CHECKPOINT)
-static const char expected_block_hashes_hash[] = "b2da201879dc7a14bcb283875a9608d465b247fdea96fc80a2972b5063259591";
+static const char expected_block_hashes_hash[] = "af0467d5d8ac1ad7232ebfd8610ec775df3438cc3b7e8033c10ad71874b72e15";
 void Blockchain::load_compiled_in_block_hashes(const GetCheckpointsCallback& get_checkpoints)
 {
  if (get_checkpoints == nullptr || !m_fast_sync)
@@ -57,6 +57,7 @@
 #include "rpc/core_rpc_server_commands_defs.h"
 #include "cryptonote_basic/difficulty.h"
 #include "cryptonote_tx_utils.h"
+#include "tx_verification_utils.h"
 #include "cryptonote_basic/verification_context.h"
 #include "crypto/hash.h"
 #include "checkpoints/checkpoints.h"
@@ -596,6 +597,15 @@ namespace cryptonote
     */
    bool store_blockchain();

+    /**
+     * @brief expands v2 transaction data from blockchain
+     *
+     * RingCT transactions do not transmit some of their data if it
+     * can be reconstituted by the receiver. This function expands
+     * that implicit data.
+     */
+    static bool expand_transaction_2(transaction &tx, const crypto::hash &tx_prefix_hash, const std::vector<std::vector<rct::ctkey>> &pubkeys);
+
    /**
     * @brief validates a transaction's inputs
     *
@@ -1222,6 +1232,9 @@ namespace cryptonote
    uint64_t m_prepare_nblocks;
    std::vector<block> *m_prepare_blocks;

+    // cache for verifying transaction RCT non semantics
+    mutable rct_ver_cache_t m_rct_ver_cache;
+
    /**
     * @brief collects the keys for all outputs being "spent" as an input
     *
@@ -1574,15 +1587,6 @@ namespace cryptonote
     */
    void load_compiled_in_block_hashes(const GetCheckpointsCallback& get_checkpoints);

-    /**
-     * @brief expands v2 transaction data from blockchain
-     *
-     * RingCT transactions do not transmit some of their data if it
-     * can be reconstituted by the receiver. This function expands
-     * that implicit data.
-     */
-    bool expand_transaction_2(transaction &tx, const crypto::hash &tx_prefix_hash, const std::vector<std::vector<rct::ctkey>> &pubkeys) const;
-
    /**
     * @brief invalidates any cached block template
     */
@@ -437,6 +437,8 @@ namespace cryptonote
    if (!sort_tx_extra(tx.extra, tx.extra))
      return false;

+    CHECK_AND_ASSERT_MES(tx.extra.size() <= MAX_TX_EXTRA_SIZE, false, "TX extra size (" << tx.extra.size() << ") is greater than max allowed (" << MAX_TX_EXTRA_SIZE << ")");
+
    //check money
    if(summary_outs_money > summary_inputs_money )
    {
@@ -219,6 +219,15 @@ namespace cryptonote
      return false;
    }

+    size_t tx_extra_size = tx.extra.size();
+    if (!kept_by_block && tx_extra_size > MAX_TX_EXTRA_SIZE)
+    {
+      LOG_PRINT_L1("transaction tx-extra is too big: " << tx_extra_size << " bytes, the limit is: " << MAX_TX_EXTRA_SIZE);
+      tvc.m_verifivation_failed = true;
+      tvc.m_tx_extra_too_big = true;
+      return false;
+    }
+
    // if the transaction came from a block popped from the chain,
    // don't check if we have its key images as spent.
    // TODO: Investigate why not?
@@ -0,0 +1,167 @@
+// Copyright (c) 2023, The Monero Project
+//
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+//
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+//
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+//
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#include "cryptonote_core/blockchain.h"
+#include "cryptonote_core/tx_verification_utils.h"
+#include "ringct/rctSigs.h"
+
+#undef MONERO_DEFAULT_LOG_CATEGORY
+#define MONERO_DEFAULT_LOG_CATEGORY "blockchain"
+
+#define VER_ASSERT(cond, msgexpr) CHECK_AND_ASSERT_MES(cond, false, msgexpr)
+
+using namespace cryptonote;
+
+// Do RCT expansion, then do post-expansion sanity checks, then do full non-semantics verification.
+static bool expand_tx_and_ver_rct_non_sem(transaction& tx, const rct::ctkeyM& mix_ring)
+{
+    // Pruned transactions can not be expanded and verified because they are missing RCT data
+    VER_ASSERT(!tx.pruned, "Pruned transaction will not pass verRctNonSemanticsSimple");
+
+    // Calculate prefix hash
+    const crypto::hash tx_prefix_hash = get_transaction_prefix_hash(tx);
+
+    // Expand mixring, tx inputs, tx key images, prefix hash message, etc into the RCT sig
+    const bool exp_res = Blockchain::expand_transaction_2(tx, tx_prefix_hash, mix_ring);
+    VER_ASSERT(exp_res, "Failed to expand rct signatures!");
+
+    const rct::rctSig& rv = tx.rct_signatures;
+
+    // Check that expanded RCT mixring == input mixring
+    VER_ASSERT(rv.mixRing == mix_ring, "Failed to check ringct signatures: mismatched pubkeys/mixRing");
+
+    // Check CLSAG/MLSAG size against transaction input
+    const size_t n_sigs = rct::is_rct_clsag(rv.type) ? rv.p.CLSAGs.size() : rv.p.MGs.size();
+    VER_ASSERT(n_sigs == tx.vin.size(), "Failed to check ringct signatures: mismatched input sigs/vin sizes");
+
+    // For each input, check that the key images were copied into the expanded RCT sig correctly
+    for (size_t n = 0; n < n_sigs; ++n)
+    {
+        const crypto::key_image& nth_vin_image = boost::get<txin_to_key>(tx.vin[n]).k_image;
+
+        if (rct::is_rct_clsag(rv.type))
+        {
+            const bool ki_match = 0 == memcmp(&nth_vin_image, &rv.p.CLSAGs[n].I, 32);
+            VER_ASSERT(ki_match, "Failed to check ringct signatures: mismatched CLSAG key image");
+        }
+        else
+        {
+            const bool mg_nonempty = !rv.p.MGs[n].II.empty();
+            VER_ASSERT(mg_nonempty, "Failed to check ringct signatures: missing MLSAG key image");
+            const bool ki_match = 0 == memcmp(&nth_vin_image, &rv.p.MGs[n].II[0], 32);
+            VER_ASSERT(ki_match, "Failed to check ringct signatures: mismatched MLSAG key image");
+        }
+    }
+
+    // Mix ring data is now known to be correctly incorporated into the RCT sig inside tx.
+    return rct::verRctNonSemanticsSimple(rv);
+}
+
+// Create a unique identifier for pair of tx blob + mix ring
+static crypto::hash calc_tx_mixring_hash(const transaction& tx, const rct::ctkeyM& mix_ring)
+{
+    std::stringstream ss;
+
+    // Start with domain seperation
+    ss << config::HASH_KEY_TXHASH_AND_MIXRING;
+
+    // Then add TX hash
+    const crypto::hash tx_hash = get_transaction_hash(tx);
+    ss.write(tx_hash.data, sizeof(crypto::hash));
+
+    // Then serialize mix ring
+    binary_archive<true> ar(ss);
+    ::do_serialize(ar, const_cast<rct::ctkeyM&>(mix_ring));
+
+    // Calculate hash of TX hash and mix ring blob
+    crypto::hash tx_and_mixring_hash;
+    get_blob_hash(ss.str(), tx_and_mixring_hash);
+
+    return tx_and_mixring_hash;
+}
+
+////////////////////////////////////////////////////////////////////////////////////////////////////
+
+namespace cryptonote
+{
+
+bool ver_rct_non_semantics_simple_cached
+(
+    transaction& tx,
+    const rct::ctkeyM& mix_ring,
+    rct_ver_cache_t& cache,
+    const std::uint8_t rct_type_to_cache
+)
+{
+    // Hello future Monero dev! If you got this assert, read the following carefully:
+    //
+    // For this version of RCT, the way we guaranteed that verification caches do not generate false
+    // positives (and thus possibly enabling double spends) is we take a hash of two things. One,
+    // we use get_transaction_hash() which gives us a (cryptographically secure) unique
+    // representation of all "knobs" controlled by the possibly malicious constructor of the
+    // transaction. Two, we take a hash of all *previously validated* blockchain data referenced by
+    // this transaction which is required to validate the ring signature. In our case, this is the
+    // mixring. Future versions of the protocol may differ in this regard, but if this assumptions
+    // holds true in the future, enable the verification hash by modifying the `untested_tx`
+    // condition below.
+    const bool untested_tx = tx.version > 2 || tx.rct_signatures.type > rct::RCTTypeBulletproofPlus;
+    VER_ASSERT(!untested_tx, "Unknown TX type. Make sure RCT cache works correctly with this type and then enable it in the code here.");
+
+    // Don't cache older (or newer) rctSig types
+    // This cache only makes sense when it caches data from mempool first,
+    // so only "current fork version-enabled" RCT types need to be cached
+    if (tx.rct_signatures.type != rct_type_to_cache)
+    {
+        MDEBUG("RCT cache: tx " << get_transaction_hash(tx) << " skipped");
+        return expand_tx_and_ver_rct_non_sem(tx, mix_ring);
+    }
+
+    // Generate unique hash for tx+mix_ring pair
+    const crypto::hash tx_mixring_hash = calc_tx_mixring_hash(tx, mix_ring);
+
+    // Search cache for successful verification of same TX + mix ring combination
+    if (cache.has(tx_mixring_hash))
+    {
+        MDEBUG("RCT cache: tx " << get_transaction_hash(tx) << " hit");
+        return true;
+    }
+
+    // We had a cache miss, so now we must expand the mix ring and do full verification
+    MDEBUG("RCT cache: tx " << get_transaction_hash(tx) << " missed");
+    if (!expand_tx_and_ver_rct_non_sem(tx, mix_ring))
+    {
+        return false;
+    }
+
+    // At this point, the TX RCT verified successfully, so add it to the cache and return true
+    cache.add(tx_mixring_hash);
+
+    return true;
+}
+
+} // namespace cryptonote
@@ -0,0 +1,78 @@
+// Copyright (c) 2023, The Monero Project
+//
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+//
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+//
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+//
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#pragma once
+
+#include "common/data_cache.h"
+#include "cryptonote_basic/cryptonote_basic.h"
+
+namespace cryptonote
+{
+
+// Modifying this value should not affect consensus. You can adjust it for performance needs
+static constexpr const size_t RCT_VER_CACHE_SIZE = 8192;
+
+using rct_ver_cache_t = ::tools::data_cache<::crypto::hash, RCT_VER_CACHE_SIZE>;
+
+/**
+ * @brief Cached version of rct::verRctNonSemanticsSimple
+ *
+ * This function will not affect how the transaction is serialized and it will never modify the
+ * transaction prefix.
+ *
+ * The reference to tx is mutable since the transaction's ring signatures may be expanded by
+ * Blockchain::expand_transaction_2. However, on cache hits, the transaction will not be
+ * expanded. This means that the caller does not need to call expand_transaction_2 on this
+ * transaction before passing it; the transaction will not successfully verify with "old" RCT data
+ * if the transaction has been otherwise modified since the last verification.
+ *
+ * But, if cryptonote::get_transaction_hash(tx) returns a "stale" hash, this function is not
+ * guaranteed to work. So make sure that the cryptonote::transaction passed has not had
+ * modifications to it since the last time its hash was fetched without properly invalidating the
+ * hashes.
+ *
+ * rct_type_to_cache can be any RCT version value as long as rct::verRctNonSemanticsSimple works for
+ * this RCT version, but for most applications, it doesn't make sense to not make this version
+ * the "current" RCT version (i.e. the version that transactions in the mempool are).
+ *
+ * @param tx transaction which contains RCT signature to verify
+ * @param mix_ring mixring referenced by this tx. THIS DATA MUST BE PREVIOUSLY VALIDATED
+ * @param cache saves tx+mixring hashes used to cache calls
+ * @param rct_type_to_cache Only RCT sigs with version (e.g. RCTTypeBulletproofPlus) will be cached
+ * @return true when verRctNonSemanticsSimple() w/ expanded tx.rct_signatures would return true
+ * @return false when verRctNonSemanticsSimple() w/ expanded tx.rct_signatures would return false
+ */
+bool ver_rct_non_semantics_simple_cached
+(
+    transaction& tx,
+    const rct::ctkeyM& mix_ring,
+    rct_ver_cache_t& cache,
+    std::uint8_t rct_type_to_cache
+);
+
+} // namespace cryptonote
@@ -526,6 +526,7 @@ namespace hw {
        {0x2c97, 0x0001, 0, 0xffa0}, 
        {0x2c97, 0x0004, 0, 0xffa0},       
        {0x2c97, 0x0005, 0, 0xffa0},
+        {0x2c97, 0x0006, 0, 0xffa0},
    };

    bool device_ledger::connect(void) {
@@ -247,7 +247,23 @@ namespace nodetool
    if (it == m_blocked_hosts.end())
    {
      m_blocked_hosts[host_str] = limit;
-      added = true;
+
+      // if the host was already blocked due to being in a blocked subnet, let it be silent
+      bool matches_blocked_subnet = false;
+      if (addr.get_type_id() == epee::net_utils::address_type::ipv4)
+      {
+        auto ipv4_address = addr.template as<epee::net_utils::ipv4_network_address>();
+        for (auto jt = m_blocked_subnets.begin(); jt != m_blocked_subnets.end(); ++jt)
+        {
+          if (jt->first.matches(ipv4_address))
+          {
+            matches_blocked_subnet = true;
+            break;
+          }
+        }
+      }
+      if (!matches_blocked_subnet)
+        added = true;
    }
    else if (it->second < limit || !add_only)
      it->second = limit;
@@ -317,6 +333,7 @@ namespace nodetool
      limit = std::numeric_limits<time_t>::max();
    else
      limit = now + seconds;
+    const bool added = m_blocked_subnets.find(subnet) == m_blocked_subnets.end();
    m_blocked_subnets[subnet] = limit;

    // drop any connection to that subnet. This should only have to look into
@@ -349,7 +366,10 @@ namespace nodetool
      conns.clear();
    }

-    MCLOG_CYAN(el::Level::Info, "global", "Subnet " << subnet.host_str() << " blocked.");
+    if (added)
+      MCLOG_CYAN(el::Level::Info, "global", "Subnet " << subnet.host_str() << " blocked.");
+    else
+      MINFO("Subnet " << subnet.host_str() << " blocked.");
    return true;
  }
  //-----------------------------------------------------------------------------------
@@ -30,7 +30,6 @@

 #include "misc_log_ex.h"
 #include "misc_language.h"
-#include "common/data_cache.h"
 #include "common/perf_timer.h"
 #include "common/threadpool.h"
 #include "common/util.h"
@@ -1579,42 +1578,6 @@ namespace rct {
      }
    }

-    bool verRctNonSemanticsSimpleCached(const rctSig & rv)
-    {
-      // Hello future Monero dev! If you got this assert, read the following carefully:
-      //
-      // RCT cache assumes that this function will serialize and hash all rv's fields used for RingCT verification
-      // If you're about to add a new RCTType here, first you must check that binary_archive serialization writes all rv's fields to the binary blob
-      // If it's not the case, rewrite this function to serialize everything, even some "temporary" fields which are not serialized normally
-      CHECK_AND_ASSERT_MES_L1(rv.type <= RCTTypeBulletproofPlus, false, "Unknown RCT type. Make sure RCT cache works correctly with this type and then enable it in the code here.");
-
-      // Don't cache older (or newer) rctSig types
-      // This cache only makes sense when it caches data from mempool first,
-      // so only "current fork version-enabled" RCT types need to be cached
-      if (rv.type != RCTTypeBulletproofPlus)
-        return verRctNonSemanticsSimple(rv);
-
-      // Get the hash of rv
-      std::stringstream ss;
-      binary_archive<true> ar(ss);
-
-      ::do_serialize(ar, const_cast<rctSig&>(rv));
-
-      crypto::hash h;
-      cryptonote::get_blob_hash(ss.str(), h);
-
-      static tools::data_cache<crypto::hash, 8192> cache;
-
-      if (cache.has(h))
-        return true;
-
-      const bool res = verRctNonSemanticsSimple(rv);
-      if (res)
-        cache.add(h);
-
-      return res;
-    }
-
    //RingCT protocol
    //genRct: 
    //   creates an rctSig with all data necessary to verify the rangeProofs and that the signer owns one of the
@@ -132,7 +132,6 @@ namespace rct {
    bool verRctSemanticsSimple(const rctSig & rv);
    bool verRctSemanticsSimple(const std::vector<const rctSig*> & rv);
    bool verRctNonSemanticsSimple(const rctSig & rv);
-    bool verRctNonSemanticsSimpleCached(const rctSig & rv);
    static inline bool verRctSimple(const rctSig & rv) { return verRctSemanticsSimple(rv) && verRctNonSemanticsSimple(rv); }
    xmr_amount decodeRct(const rctSig & rv, const key & sk, unsigned int i, key & mask, hw::device &hwdev);
    xmr_amount decodeRct(const rctSig & rv, const key & sk, unsigned int i, hw::device &hwdev);
@@ -97,6 +97,14 @@ namespace rct {
    struct ctkey {
        key dest;
        key mask; //C here if public
+
+        bool operator==(const ctkey &other) const {
+          return (dest == other.dest) && (mask == other.mask);
+        }
+
+        bool operator!=(const ctkey &other) const {
+          return !(*this == other);
+        }
    };
    typedef std::vector<ctkey> ctkeyV;
    typedef std::vector<ctkeyV> ctkeyM;
@@ -1275,6 +1275,8 @@ namespace cryptonote
          add_reason(reason, "fee too low");
        if ((res.too_few_outputs = tvc.m_too_few_outputs))
          add_reason(reason, "too few outputs");
+        if ((res.tx_extra_too_big = tvc.m_tx_extra_too_big))
+          add_reason(reason, "tx-extra too big");
        const std::string punctuation = reason.empty() ? "" : ": ";
        if (tvc.m_verifivation_failed)
        {
@@ -88,7 +88,7 @@ namespace cryptonote
 // advance which version they will stop working with
 // Don't go over 32767 for any of these
 #define CORE_RPC_VERSION_MAJOR 3
-#define CORE_RPC_VERSION_MINOR 11
+#define CORE_RPC_VERSION_MINOR 12
 #define MAKE_CORE_RPC_VERSION(major,minor) (((major)<<16)|(minor))
 #define CORE_RPC_VERSION MAKE_CORE_RPC_VERSION(CORE_RPC_VERSION_MAJOR, CORE_RPC_VERSION_MINOR)

@@ -592,6 +592,7 @@ namespace cryptonote
      bool fee_too_low;
      bool too_few_outputs;
      bool sanity_check_failed;
+      bool tx_extra_too_big;

      BEGIN_KV_SERIALIZE_MAP()
        KV_SERIALIZE_PARENT(rpc_access_response_base)
@@ -606,6 +607,7 @@ namespace cryptonote
        KV_SERIALIZE(fee_too_low)
        KV_SERIALIZE(too_few_outputs)
        KV_SERIALIZE(sanity_check_failed)
+        KV_SERIALIZE(tx_extra_too_big)
      END_KV_SERIALIZE_MAP()
    };
    typedef epee::misc_utils::struct_init<response_t> response;
@@ -1,5 +1,5 @@
 #define DEF_MONERO_VERSION_TAG "@VERSIONTAG@"
-#define DEF_MONERO_VERSION "0.18.2.0"
+#define DEF_MONERO_VERSION "0.18.2.1"
 #define DEF_MONERO_RELEASE_NAME "Fluorine Fermi"
 #define DEF_MONERO_VERSION_FULL DEF_MONERO_VERSION "-" DEF_MONERO_VERSION_TAG
 #define DEF_MONERO_VERSION_IS_RELEASE @VERSION_IS_RELEASE@
@@ -1005,7 +1005,7 @@ gamma_picker::gamma_picker(const std::vector<uint64_t> &rct_offsets, double shap
  const size_t blocks_to_consider = std::min<size_t>(rct_offsets.size(), blocks_in_a_year);
  const size_t outputs_to_consider = rct_offsets.back() - (blocks_to_consider < rct_offsets.size() ? rct_offsets[rct_offsets.size() - blocks_to_consider - 1] : 0);
  begin = rct_offsets.data();
-  end = rct_offsets.data() + rct_offsets.size() - CRYPTONOTE_DEFAULT_TX_SPENDABLE_AGE;
+  end = rct_offsets.data() + rct_offsets.size() - (std::max(1, CRYPTONOTE_DEFAULT_TX_SPENDABLE_AGE) - 1);
  num_rct_outputs = *(end - 1);
  THROW_WALLET_EXCEPTION_IF(num_rct_outputs == 0, error::wallet_internal_error, "No rct outputs");
  average_output_time = DIFFICULTY_TARGET_V2 * blocks_to_consider / static_cast<double>(outputs_to_consider); // this assumes constant target over the whole rct range
@@ -8408,7 +8408,7 @@ void wallet2::get_outs(std::vector<std::vector<tools::wallet2::get_outs_entry>>
      else
      {
        // the base offset of the first rct output in the first unlocked block (or the one to be if there's none)
-        num_outs = rct_offsets[rct_offsets.size() - CRYPTONOTE_DEFAULT_TX_SPENDABLE_AGE];
+        num_outs = gamma->get_num_rct_outs();
        LOG_PRINT_L1("" << num_outs << " unlocked rct outputs");
        THROW_WALLET_EXCEPTION_IF(num_outs == 0, error::wallet_internal_error,
            "histogram reports no unlocked rct outputs, not even ours");
@@ -8692,7 +8692,7 @@ void wallet2::get_outs(std::vector<std::vector<tools::wallet2::get_outs_entry>>
      }
      bool use_histogram = amount != 0;
      if (!use_histogram)
-        num_outs = rct_offsets[rct_offsets.size() - CRYPTONOTE_DEFAULT_TX_SPENDABLE_AGE];
+        num_outs = gamma->get_num_rct_outs();

      // make sure the real outputs we asked for are really included, along
      // with the correct key and mask: this guards against an active attack
@@ -101,6 +101,7 @@ namespace tools
    uint64_t pick();
    gamma_picker(const std::vector<uint64_t> &rct_offsets);
    gamma_picker(const std::vector<uint64_t> &rct_offsets, double shape, double scale);
+    uint64_t get_num_rct_outs() const { return num_rct_outputs; }

  private:
    struct gamma_engine
@@ -91,6 +91,7 @@ set(unit_tests_sources
  unbound.cpp
  uri.cpp
  varint.cpp
+  ver_rct_non_semantics_simple_cached.cpp
  ringct.cpp
  output_selection.cpp
  vercmp.cpp
@@ -0,0 +1,426 @@
+// Copyright (c) 2023, The Monero Project
+//
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+//
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+//
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+//
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#include <sstream>
+
+#define IN_UNIT_TESTS // To access Blockchain::{expand_transaction_2, verRctNonSemanticsSimpleCached}
+
+#include "gtest/gtest.h"
+#include "unit_tests_utils.h"
+
+#include "cryptonote_basic/cryptonote_format_utils.h"
+#include "cryptonote_core/blockchain.h"
+#include "file_io_utils.h"
+#include "misc_log_ex.h"
+#include "ringct/rctSigs.h"
+
+namespace cryptonote
+{
+// declaration not provided in cryptonote_format_utils.h, but definition is not static ;)
+bool expand_transaction_1(transaction &tx, bool base_only);
+}
+
+namespace
+{
+/**
+ * @brief Make rct::ctkey from hex string representation of destionation and mask
+ *
+ * @param dest_hex
+ * @param mask_hex
+ * @return rct::ctkey
+ */
+static rct::ctkey make_ctkey(const char* dest_hex, const char* mask_hex)
+{
+    rct::key dest;
+    rct::key mask;
+    CHECK_AND_ASSERT_THROW_MES(epee::from_hex::to_buffer(epee::as_mut_byte_span(dest), dest_hex), "dest bad hex: " << dest_hex);
+    CHECK_AND_ASSERT_THROW_MES(epee::from_hex::to_buffer(epee::as_mut_byte_span(mask), mask_hex), "mask bad hex: " << mask_hex);
+    return {dest, mask};
+}
+
+template <typename T>
+static std::string stringify_with_do_serialize(const T& t)
+{
+    std::stringstream ss;
+    binary_archive<true> ar(ss);
+    CHECK_AND_ASSERT_THROW_MES(ar.good(), "Archiver is not in a good state. This shouldn't happen!");
+    ::do_serialize(ar, const_cast<T&>(t));
+    return ss.str();
+}
+
+static bool check_tx_is_expanded(const cryptonote::transaction& tx, const rct::ctkeyM& pubkeys)
+{
+    // Ripped from cryptonote_core/blockchain.cpp
+
+    const rct::rctSig& rv = tx.rct_signatures;
+
+    if (pubkeys.size() != rv.mixRing.size())
+    {
+        MERROR("Failed to check ringct signatures: mismatched pubkeys/mixRing size");
+        return false;
+    }
+    for (size_t i = 0; i < pubkeys.size(); ++i)
+    {
+        if (pubkeys[i].size() != rv.mixRing[i].size())
+        {
+            MERROR("Failed to check ringct signatures: mismatched pubkeys/mixRing size");
+            return false;
+        }
+    }
+
+    for (size_t n = 0; n < pubkeys.size(); ++n)
+    {
+        for (size_t m = 0; m < pubkeys[n].size(); ++m)
+        {
+            if (pubkeys[n][m].dest != rct::rct2pk(rv.mixRing[n][m].dest))
+            {
+                MERROR("Failed to check ringct signatures: mismatched pubkey at vin " << n << ", index " << m);
+                return false;
+            }
+            if (pubkeys[n][m].mask != rct::rct2pk(rv.mixRing[n][m].mask))
+            {
+                MERROR("Failed to check ringct signatures: mismatched commitment at vin " << n << ", index " << m);
+                return false;
+            }
+        }
+    }
+
+    const size_t n_sigs = rct::is_rct_clsag(rv.type) ? rv.p.CLSAGs.size() : rv.p.MGs.size();
+    if (n_sigs != tx.vin.size())
+    {
+        MERROR("Failed to check ringct signatures: mismatched MGs/vin sizes");
+        return false;
+    }
+    for (size_t n = 0; n < tx.vin.size(); ++n)
+    {
+        bool error;
+        if (rct::is_rct_clsag(rv.type))
+            error = memcmp(&boost::get<cryptonote::txin_to_key>(tx.vin[n]).k_image, &rv.p.CLSAGs[n].I, 32);
+        else
+            error = rv.p.MGs[n].II.empty() || memcmp(&boost::get<cryptonote::txin_to_key>(tx.vin[n]).k_image, &rv.p.MGs[n].II[0], 32);
+        if (error)
+        {
+            MERROR("Failed to check ringct signatures: mismatched key image");
+            return false;
+        }
+    }
+
+    return true;
+}
+
+/**
+ * @brief Perform expand_transaction_1 and Blockchain::expand_transaction_2 on a certain transaction
+ */
+static void expand_transaction_fully(cryptonote::transaction& tx, const rct::ctkeyM& input_pubkeys)
+{
+    const crypto::hash tx_prefix_hash = cryptonote::get_transaction_prefix_hash(tx);
+    CHECK_AND_ASSERT_THROW_MES(cryptonote::expand_transaction_1(tx, false), "expand 1 failed");
+    CHECK_AND_ASSERT_THROW_MES
+    (
+        cryptonote::Blockchain::expand_transaction_2(tx, tx_prefix_hash, input_pubkeys),
+        "expand 2 failed"
+    );
+    CHECK_AND_ASSERT_THROW_MES(!memcmp(&tx_prefix_hash, &tx.rct_signatures.message, 32), "message check failed");
+    CHECK_AND_ASSERT_THROW_MES(input_pubkeys == tx.rct_signatures.mixRing, "mixring check failed");
+    CHECK_AND_ASSERT_THROW_MES(check_tx_is_expanded(tx, input_pubkeys), "tx expansion check 2 failed");
+}
+
+/**
+ * @brief Mostly construct transaction from binary file and provided mix ring pubkeys
+ *
+ * Most important to us, this should populate the .rct_signatures.message and
+ * .rct_signatures.mixRings fields of the transaction.
+ *
+ * @param file_name relative file path in unit test data directory
+ * @param input_pubkeys manually retrived input pubkey destination / masks for each ring
+ * @return cryptonote::transaction the expanded transaction
+ */
+static cryptonote::transaction expand_transaction_from_bin_file_and_pubkeys
+(
+    const char* file_name,
+    const rct::ctkeyM& input_pubkeys
+)
+{
+    cryptonote::transaction transaction;
+
+    const boost::filesystem::path tx_json_path = unit_test::data_dir / file_name;
+    std::string tx_blob;
+    CHECK_AND_ASSERT_THROW_MES
+    (
+        epee::file_io_utils::load_file_to_string(tx_json_path.string(), tx_blob),
+        "loading file to string failed"
+    );
+
+    CHECK_AND_ASSERT_THROW_MES
+    (
+        cryptonote::parse_and_validate_tx_from_blob(tx_blob, transaction),
+        "TX blob could not be parsed"
+    );
+
+    expand_transaction_fully(transaction, input_pubkeys);
+
+    return transaction;
+}
+
+/**
+ * @brief Return whether a modification changes blob resulting from do_serialize()
+ */
+template <typename T, class TModifier>
+static bool modification_changes_do_serialize
+(
+    const T& og_obj,
+    TModifier& obj_modifier_func,
+    bool expected_change
+)
+{
+    T modded_obj = og_obj;
+    obj_modifier_func(modded_obj);
+    const std::string og_blob = stringify_with_do_serialize(og_obj);
+    const std::string modded_blob = stringify_with_do_serialize(modded_obj);
+    const bool did_change = modded_blob != og_blob;
+    if (did_change != expected_change)
+    {
+        const std::string og_hex = epee::to_hex::string(epee::strspan<uint8_t>(og_blob));
+        const std::string modded_hex = epee::to_hex::string(epee::strspan<uint8_t>(modded_blob));
+        MERROR("unexpected: modded_blob '" << modded_hex << "' vs og_blob ' << " << og_hex << "'");
+    }
+    return did_change;
+}
+
+// Contains binary representation of mainnet transaction (height 2777777):
+// e89415b95564aa7e3587c91422756ba5303e727996e19c677630309a0d52a7ca
+static constexpr const char* tx1_file_name = "txs/bpp_tx_e89415.bin";
+
+// This contains destination key / mask pairs for each output in the input ring of the above tx
+static const rct::ctkeyM tx1_input_pubkeys =
+{{
+    make_ctkey("e50f476129d40af31e0938743f7f2d60e867aab31294f7acaf6e38f0976f0228", "51e788ddf5c95c124a7314d45a91b52d60db25a0572de9c2b4ec515aca3d4481"),
+    make_ctkey("804245d067fcfe6cd66376db0571869989bc68b3e22a0f902109c7530df47a59", "c3cc65d3b3a05defaa05213dc3b0496f9b86dbeeefbff28db34b134b6ee3230b"),
+    make_ctkey("527563a03b498e47732b815f5f0c5875a70e0fb71a37c88123f0f8686349fae4", "04417c03b397cd11e403275ec89cb0ab5b8476bb88470e9ae7208ea63dacf073"),
+    make_ctkey("bffca8b5c7fe4235ba7136d6b5325f63df343dc147940b677f50217f8953bca6", "5cd8c5e54e07275422c9c5a9f4a7268d26c494ffba419e878b7e873a02ae2e76"),
+    make_ctkey("1f73385ea74308aa78b5abf585faac14a5e78a6e23f0f68c9c14681108b28ef0", "5c02b3156daaa8ec476d3244439d90efa266f3e51cb9c8eb384d8b9a8efaa024"),
+    make_ctkey("a2421eae8bb256644b34feeab48c6086c2c9feb40d2643436dc45e303eee8ab2", "787823abffa988b56d4a7b4a834630f71520220fd82fad035955e616ec095788"),
+    make_ctkey("17d8d8dc1e1c25b7295f2eab44c4ccc08a629b8e8d781bbb6f9a51a9561bcd4c", "db1ea24be6947e03176a297160dba16d65f37751bb0ef2ba71a4590d12b61dfc"),
+    make_ctkey("2c39348a9ab04dbabe3b5249819b7845ed8aaebd0d8eddd98bda0bf40753a398", "4e6cd25fbd10e2e040be84e3bf8043c612daeef625e66a5e5bcff88c9c46e82c"),
+    make_ctkey("c4c97157f23b45c7084526aaa9958fe858bebe446a7efa22c491c439b74271b1", "e251db2c86193a11a5bffefffe48c20e3d92a8dc98cb3a2f41704e565bcd860a"),
+    make_ctkey("d342045525139a8551bcdfa7aa0117d2ac2327cb6cf449ca59420c300e4471a5", "789c11f72060ad80f4cda5d89b24d49f9435bf765598dea7a91776e99f05f87c"),
+    make_ctkey("9a972ccf2c74f648070b0be839749c98eca87166de401a6c1f59e64b938a46c1", "5444cbed5cec31fb6ed1612f815d292f2bf3d2ff584bbcd8e5201ec59670d414"),
+    make_ctkey("49ccb806ccf5cbd74bae8d9fb2da8918ab61d0774ee6a6c3a6ccd237db22a088", "0c5db942fb44f29f6ef956e24db91f98a6de6e7288b0b04d01b8f260453d1431"),
+    make_ctkey("74417e8d1483df2df6fe68c88fc9a72639c35d765b38351b838521addf45dadc", "a1a606d6c4762ef51c1759bcb8b5c88be1d323025400c41fe6885431064b64dc"),
+    make_ctkey("48c4c349adaf7b3be27656ea70d1c83b93e1511bb0aac987861a4da9689b0e95", "ad14ffd5edac199ea7c5437d558089b0f2f03aa74bde43611322d769968b5a1c"),
+    make_ctkey("2d2ffade0f85ddd83a036469e49542e93cad94f9bea535f0ea2eb2f56304517e", "bcc48d00bd06dc5439200e749d0caf8a062b072d0c0eb1f78f6a4d8f2373e5f4"),
+    make_ctkey("4ee857d0ce17f66eca9c81eb326e404ceb50c8198248f2f827c440ee7aa0c0d7", "a8a9d61d4abbfb123630ffd214c834cc45113eaa51dd2f904cc6ae0c3c5d70e3")
+}};
+} // anonymous namespace
+
+TEST(verRctNonSemanticsSimple, tx1_preconditions)
+{
+    // If this unit test fails, something changed about transaction deserialization / expansion or
+    // something changed about RingCT signature verification.
+
+    cryptonote::rct_ver_cache_t rct_ver_cache;
+
+    cryptonote::transaction tx = expand_transaction_from_bin_file_and_pubkeys
+        (tx1_file_name, tx1_input_pubkeys);
+    const rct::rctSig& rs = tx.rct_signatures;
+
+    const crypto::hash tx_prefix_hash = cryptonote::get_transaction_prefix_hash(tx);
+
+    EXPECT_EQ(1, tx.vin.size());
+    EXPECT_EQ(2, tx.vout.size());
+    const rct::key expected_sig_msg = rct::hash2rct(tx_prefix_hash);
+    EXPECT_EQ(expected_sig_msg, rs.message);
+    EXPECT_EQ(1, rs.mixRing.size());
+    EXPECT_EQ(16, rs.mixRing[0].size());
+    EXPECT_EQ(0, rs.pseudoOuts.size());
+    EXPECT_EQ(0, rs.p.rangeSigs.size());
+    EXPECT_EQ(0, rs.p.bulletproofs.size());
+    EXPECT_EQ(1, rs.p.bulletproofs_plus.size());
+    EXPECT_EQ(2, rs.p.bulletproofs_plus[0].V.size());
+    EXPECT_EQ(7, rs.p.bulletproofs_plus[0].L.size());
+    EXPECT_EQ(7, rs.p.bulletproofs_plus[0].R.size());
+    EXPECT_EQ(0, rs.p.MGs.size());
+    EXPECT_EQ(1, rs.p.CLSAGs.size());
+    EXPECT_EQ(16, rs.p.CLSAGs[0].s.size());
+    EXPECT_EQ(1, rs.p.pseudoOuts.size());
+    EXPECT_EQ(tx1_input_pubkeys, rs.mixRing);
+    EXPECT_EQ(2, rs.outPk.size());
+
+    EXPECT_TRUE(rct::verRctSemanticsSimple(rs));
+    EXPECT_TRUE(rct::verRctNonSemanticsSimple(rs));
+    EXPECT_TRUE(rct::verRctSimple(rs));
+    EXPECT_TRUE(cryptonote::ver_rct_non_semantics_simple_cached(tx, tx1_input_pubkeys, rct_ver_cache, rct::RCTTypeBulletproofPlus));
+    EXPECT_TRUE(cryptonote::ver_rct_non_semantics_simple_cached(tx, tx1_input_pubkeys, rct_ver_cache, rct::RCTTypeBulletproofPlus));
+}
+
+#define SERIALIZABLE_SIG_CHANGES_SUBTEST(fieldmodifyclause)                                    \
+    do {                                                                                       \
+        const auto sig_modifier_func = [](rct::rctSig& rs) { rs.fieldmodifyclause; };          \
+        EXPECT_TRUE(modification_changes_do_serialize(original_sig, sig_modifier_func, true)); \
+    } while (0);                                                                               \
+
+TEST(verRctNonSemanticsSimple, serializable_sig_changes)
+{
+    // Hello, future visitors. If this unit test fails, then fields of rctSig have been dropped from
+    // serialization.
+
+    const cryptonote::transaction tx = expand_transaction_from_bin_file_and_pubkeys
+        (tx1_file_name, tx1_input_pubkeys);
+    const rct::rctSig& original_sig = tx.rct_signatures;
+
+    // These are the subtests most likely to fail. Fields 'message' and 'mixRing' are not serialized
+    // when sent over the wire, since they can be reconstructed from transaction data. However, they
+    // are serialized by ::do_serialize(rctSig).
+    // How signatures are serialized for the blockchain can be found in the methods
+    // rct::rctSigBase::serialize_rctsig_base and rct::rctSigPrunable::serialize_rctsig_prunable.
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(message.bytes[31]++)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(mixRing.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(mixRing[0].push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(mixRing[0][8].dest[10]--)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(mixRing[0][15].mask[3]--)
+
+    // rctSigBase changes. These subtests are less likely to break
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(type ^= 23)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(pseudoOuts.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(ecdhInfo.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(outPk.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(outPk[0].dest[14]--)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(outPk[1].dest[14]--)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(outPk[0].mask[14]--)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(outPk[1].mask[14]--)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(txnFee *= 2023)
+
+    // rctSigPrunable changes
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.rangeSigs.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].A[13] -= 7)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].A1[13] -= 7)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].B[13] -= 7)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].r1[13] -= 7)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].s1[13] -= 7)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].d1[13] -= 7)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].L.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].L[2][13] -= 7)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].R.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].R[2][13] -= 7)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.MGs.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.CLSAGs.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.CLSAGs[0].s.push_back({}))
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.CLSAGs[0].s[15][31] ^= 69)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.CLSAGs[0].c1[0] /= 3)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.CLSAGs[0].D[0] /= 3)
+    SERIALIZABLE_SIG_CHANGES_SUBTEST(p.pseudoOuts.push_back({}))
+
+    // Uncomment line below to sanity check SERIALIZABLE_SIG_CHANGES_SUBTEST
+    // SERIALIZABLE_SIG_CHANGES_SUBTEST(message) // should fail
+}
+
+#define UNSERIALIZABLE_SIG_CHANGES_SUBTEST(fieldmodifyclause)                                    \
+    do {                                                                                         \
+        const auto sig_modifier_func = [](rct::rctSig& rs) { rs.fieldmodifyclause; };            \
+        EXPECT_FALSE(modification_changes_do_serialize(original_sig, sig_modifier_func, false)); \
+    } while (0);                                                                                 \
+
+TEST(verRctNonSemanticsSimple, unserializable_sig_changes)
+{
+    // Hello, future visitors. If this unit test fails, then congrats! ::do_serialize(rctSig) became
+    // better at uniquely representing rctSig.
+    const cryptonote::transaction tx = expand_transaction_from_bin_file_and_pubkeys
+        (tx1_file_name, tx1_input_pubkeys);
+    const rct::rctSig& original_sig = tx.rct_signatures;
+
+    UNSERIALIZABLE_SIG_CHANGES_SUBTEST(p.CLSAGs[0].I[14]++)
+    UNSERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].V.push_back({}))
+    UNSERIALIZABLE_SIG_CHANGES_SUBTEST(p.bulletproofs_plus[0].V[1][31]--)
+
+    // Uncomment line below to sanity check UNSERIALIZABLE_SIG_CHANGES_SUBTEST_SHORTCUT
+    // UNSERIALIZABLE_SIG_CHANGES_SUBTEST_SHORTCUT(message[2]++) // should fail
+}
+
+#define SERIALIZABLE_MIXRING_CHANGES_SUBTEST(fieldmodifyclause)                                   \
+    do {                                                                                          \
+        using mr_mod_func_t = std::function<void(rct::ctkeyM&)>;                                  \
+        const mr_mod_func_t mr_modifier_func = [&](rct::ctkeyM& mr) { mr fieldmodifyclause; };    \
+        EXPECT_TRUE(modification_changes_do_serialize(original_mixring, mr_modifier_func, true)); \
+    } while (0);                                                                                  \
+
+TEST(verRctNonSemanticsSimple, serializable_mixring_changes)
+{
+    // Hello, future Monero devs! If this unit test fails, a huge concensus-related assumption has
+    // been broken and verRctNonSemanticsSimpleCached needs to be reevalulated for validity. If it
+    // is not, there may be an exploit which allows for double-spending. See the implementation for
+    // more comments on the uniqueness of the internal cache hash.
+
+    const rct::ctkeyM original_mixring = tx1_input_pubkeys;
+
+    const size_t mlen = tx1_input_pubkeys.size();
+    ASSERT_EQ(1, mlen);
+    const size_t nlen = tx1_input_pubkeys[0].size();
+    ASSERT_EQ(16, nlen);
+
+    SERIALIZABLE_MIXRING_CHANGES_SUBTEST(.clear())
+    SERIALIZABLE_MIXRING_CHANGES_SUBTEST(.push_back({}))
+    SERIALIZABLE_MIXRING_CHANGES_SUBTEST([0].clear())
+    SERIALIZABLE_MIXRING_CHANGES_SUBTEST([0].push_back({}))
+    SERIALIZABLE_MIXRING_CHANGES_SUBTEST([0][0].dest[4]--)
+    SERIALIZABLE_MIXRING_CHANGES_SUBTEST([0][15].mask[31]--)
+
+    // Loop through all bytes of the mixRing and check for serialiable changes
+    for (size_t i = 0; i < mlen; ++i)
+    {
+        for (size_t j = 0; j < nlen; ++j)
+        {
+            static_assert(sizeof(rct::key) == 32, "rct::key size wrong");
+            for (size_t k = 0; k < sizeof(rct::key); ++k)
+            {
+                SERIALIZABLE_MIXRING_CHANGES_SUBTEST([i][j].dest[k]++)
+                SERIALIZABLE_MIXRING_CHANGES_SUBTEST([i][j].mask[k]++)
+            }
+        }
+    }
+}
+
+#define EXPAND_TRANSACTION_2_FAILURES_SUBTEST(fieldmodifyclause)       \
+    do {                                                               \
+        cryptonote::transaction test_tx = original_tx;                 \
+        test_tx.fieldmodifyclause;                                     \
+        test_tx.invalidate_hashes();                                   \
+        EXPECT_FALSE(check_tx_is_expanded(test_tx, original_mixring)); \
+    } while (0);                                                       \
+
+TEST(verRctNonSemanticsSimple, expand_transaction_2_failures)
+{
+    cryptonote::transaction original_tx = expand_transaction_from_bin_file_and_pubkeys
+        (tx1_file_name, tx1_input_pubkeys);
+    rct::ctkeyM original_mixring = tx1_input_pubkeys;
+
+    EXPAND_TRANSACTION_2_FAILURES_SUBTEST(rct_signatures.p.CLSAGs[0].I[0]++)
+    EXPAND_TRANSACTION_2_FAILURES_SUBTEST(rct_signatures.mixRing[0][15].dest[31]++)
+    EXPAND_TRANSACTION_2_FAILURES_SUBTEST(rct_signatures.mixRing[0][15].mask[31]++)
+}
Author	SHA1	Message	Date
luigi1111	4f47fd2626	Merge pull request #8801 `1328048` wallet2: fix infinite loop in fake out selection (Crypto City)	2023-03-27 09:17:55 -04:00
Crypto City	132804811d	wallet2: fix infinite loop in fake out selection The gamma picker and the caller code did not quite agree on the number of rct outputs available for use - by one block - which caused an infinite loop if the picker could never pick outputs from that block but already had picked all other outputs from previous blocks. Also change the range to select from using code from UkoeHB.	2023-03-25 19:26:43 +00:00
luigi1111	25645e5d23	Merge pull request #8785 `cdeb286` build: prepare v0.18.2.1 (selsta)	2023-03-24 22:55:49 -04:00
luigi1111	0e2c2ddd9c	Merge pull request #8787 `c4cfaa4` p2p: do not log to global when re-blocking a subnet (moneromooo-monero) `f0e326b` p2p: avoid spam blocking ipv4 addresses in a blocked subnet (moneromooo-monero)	2023-03-24 22:55:29 -04:00
moneromooo-monero	c4cfaa4567	p2p: do not log to global when re-blocking a subnet	2023-03-19 02:58:38 +01:00
moneromooo-monero	f0e326be58	p2p: avoid spam blocking ipv4 addresses in a blocked subnet	2023-03-19 02:58:38 +01:00
luigi1111	225e5ba571	Merge pull request #8784 `5900ed3` Add a size limit for tx_extra in tx pool (tevador)	2023-03-18 18:23:14 -04:00
luigi1111	66f57299a2	Merge pull request #8781 `c59e009` verRctNonSemanticsSimpleCached: fix fragility (Jeffrey Ryan)	2023-03-18 18:22:01 -04:00
luigi1111	d7821a02c4	Merge pull request #8779 `14de562` device: Add ledger Stax device id to device detection (Francois Beutin)	2023-03-18 18:21:25 -04:00
luigi1111	b4519c6bbd	Merge pull request #8746 `77d883e` workflows: update dependencies to fix warnings (selsta)	2023-03-18 18:20:17 -04:00
selsta	cdeb286359	build: prepare v0.18.2.1	2023-03-18 21:23:42 +01:00
tevador	5900ed3706	Add a size limit for tx_extra in tx pool	2023-03-18 20:01:58 +01:00
Jeffrey Ryan	c59e0096b6	verRctNonSemanticsSimpleCached: fix fragility	2023-03-17 18:46:34 -05:00
Francois Beutin	14de562a6f	device: Add ledger Stax device id to device detection	2023-03-17 21:27:51 +01:00
selsta	77d883e507	workflows: update dependencies to fix warnings	2023-02-20 04:11:35 +01:00