Merge pull request #5762

0a9bda81 bump version to 0.14.1.2 (Riccardo Spagni) f4f00acc update checkpoints.dat hash (Riccardo Spagni)
bump version to 0.14.1.2
2019-07-18 17:18:13 +02:00 · 2019-07-18 16:56:57 +02:00 · 2019-07-18 16:54:25 +02:00 · 2019-07-17 22:40:18 +02:00 · 2019-07-17 22:25:13 +02:00 · 2019-07-17 21:19:24 +02:00
96 changed files with 4138 additions and 3727 deletions
@@ -672,8 +672,11 @@ else()
    add_cxx_flag_if_supported(-fstack-clash-protection CXX_SECURITY_FLAGS)
  endif()

-  add_c_flag_if_supported(-mmitigate-rop C_SECURITY_FLAGS)
-  add_cxx_flag_if_supported(-mmitigate-rop CXX_SECURITY_FLAGS)
+  # Removed in GCC 9.1 (or before ?), but still accepted, so spams the output
+  if (NOT (CMAKE_C_COMPILER_ID STREQUAL "GNU" AND NOT CMAKE_C_COMPILER_VERSION VERSION_LESS 9.1))
+    add_c_flag_if_supported(-mmitigate-rop C_SECURITY_FLAGS)
+    add_cxx_flag_if_supported(-mmitigate-rop CXX_SECURITY_FLAGS)
+  endif()

  # linker
  if (NOT WIN32)
@@ -137,7 +137,7 @@ Monero uses a fixed-schedule software upgrade (hard fork) mechanism to implement
 Dates are provided in the format YYYY-MM-DD.


-| Software upgrade block height | Date       | Fork version | Minimum Monero version | Recommended Monero version | Details                                                                            |  
+| Software upgrade block height  | Date       | Fork version      | Minimum Monero version | Recommended Monero version | Details                                                                            |  
 | ------------------------------ | -----------| ----------------- | ---------------------- | -------------------------- | ---------------------------------------------------------------------------------- |
 | 1009827                        | 2016-03-22 | v2                | v0.9.4                 | v0.9.4                     | Allow only >= ringsize 3, blocktime = 120 seconds, fee-free blocksize 60 kb       |
 | 1141317                        | 2016-09-21 | v3                | v0.9.4                 | v0.10.0                    | Splits coinbase into denominations  |
@@ -147,8 +147,8 @@ Dates are provided in the format YYYY-MM-DD.
 | 1546000                        | 2018-04-06 | v7                | v0.12.0.0              | v0.12.3.0                  | Cryptonight variant 1, ringsize >= 7, sorted inputs
 | 1685555                        | 2018-10-18 | v8                | v0.13.0.0              | v0.13.0.4                  | max transaction size at half the penalty free block size, bulletproofs enabled, cryptonight variant 2, fixed ringsize [11](https://youtu.be/KOO5S4vxi0o)
 | 1686275                        | 2018-10-19 | v9                | v0.13.0.0              | v0.13.0.4                  | bulletproofs required
-| 1788000                        | 2019-03-09 | v10               | v0.14.0.0              | v0.14.0.2                  | New PoW based on Cryptonight-R, new block weight algorithm, slightly more efficient RingCT format
-| 1788720                        | 2019-03-10 | v11               | v0.14.0.0              | v0.14.0.2                  | forbid old RingCT transaction format
+| 1788000                        | 2019-03-09 | v10               | v0.14.0.0              | v0.14.1.2                  | New PoW based on Cryptonight-R, new block weight algorithm, slightly more efficient RingCT format
+| 1788720                        | 2019-03-10 | v11               | v0.14.0.0              | v0.14.1.2                  | forbid old RingCT transaction format
 | XXXXXXX                        | 2019-10-XX | XX                | XXXXXXXXX              | XXXXXXXXX                  | X

 X's indicate that these details have not been determined as of commit date.
@@ -304,7 +304,7 @@ Tested on a Raspberry Pi Zero with a clean install of minimal Raspbian Stretch (
    ```bash
    git clone https://github.com/monero-project/monero.git
    cd monero
-    git checkout tags/v0.14.1.0
+    git checkout tags/v0.14.1.2
    ```

 * Build:
@@ -421,10 +421,10 @@ application.
    cd monero
    ```

-* If you would like a specific [version/tag](https://github.com/monero-project/monero/tags), do a git checkout for that version. eg. 'v0.14.1.0'. If you don't care about the version and just want binaries from master, skip this step:
+* If you would like a specific [version/tag](https://github.com/monero-project/monero/tags), do a git checkout for that version. eg. 'v0.14.1.2'. If you don't care about the version and just want binaries from master, skip this step:
 	
    ```bash
-    git checkout v0.14.1.0
+    git checkout v0.14.1.2
    ```

 * If you are on a 64-bit system, run:
@@ -29,7 +29,7 @@
 # Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers

 # Check what commit we're on
-execute_process(COMMAND "${GIT}" rev-parse --short HEAD RESULT_VARIABLE RET OUTPUT_VARIABLE COMMIT OUTPUT_STRIP_TRAILING_WHITESPACE)
+execute_process(COMMAND "${GIT}" rev-parse --short=9 HEAD RESULT_VARIABLE RET OUTPUT_VARIABLE COMMIT OUTPUT_STRIP_TRAILING_WHITESPACE)

 if(RET)
 	# Something went wrong, set the version tag to -unknown
@@ -38,6 +38,7 @@ if(RET)
    set(VERSIONTAG "unknown")
    configure_file("src/version.cpp.in" "${TO}")
 else()
+	string(SUBSTRING ${COMMIT} 0 9 COMMIT)
 	message(STATUS "You are currently on commit ${COMMIT}")
 	
 	# Get all the tags
@@ -217,4 +217,6 @@ download-win:
 	@$(MAKE) -s HOST=x86_64-w64-mingw32 download-one
 download: download-osx download-linux download-win

+ $(foreach package,$(all_packages),$(eval $(call ext_add_stages,$(package))))
+
 .PHONY: install cached download-one download-osx download-linux download-win download check-packages check-sources
@@ -213,6 +213,14 @@ $(1): | $($(1)_cached_checksum)

 endef

+stages = fetched extracted preprocessed configured built staged postprocessed cached cached_checksum
+
+define ext_add_stages
+$(foreach stage,$(stages),
+          $(1)_$(stage): $($(1)_$(stage))
+          .PHONY: $(1)_$(stage))
+endef
+
 # These functions create the build targets for each package. They must be
 # broken down into small steps so that each part is done for all packages
 # before moving on to the next step. Otherwise, a package's info
@@ -23,3 +23,7 @@ endef
 define $(package)_stage_cmds
  $(MAKE) DESTDIR=$($(package)_staging_dir) install
 endef
+
+define $(package)_postprocess_cmds
+  rm lib/*.la
+endef
@@ -6,6 +6,7 @@ $(package)_sha256_hash=03ad85db965f8ab2d27328abcf0bc5571af6ec0a414874b2066ee3fdd

 define $(package)_set_vars
 $(package)_config_opts=--enable-static
+$(package)_config_opts=--disable-shared
 $(package)_config_opts+=--prefix=$(host_prefix)
 endef

@@ -20,3 +21,8 @@ endef
 define $(package)_stage_cmds
  $(MAKE) DESTDIR=$($(package)_staging_dir) install
 endef
+
+define $(package)_postprocess_cmds
+  rm lib/*.la
+endef
+
@@ -1,8 +1,8 @@
 package=hidapi
-$(package)_version=0.8.0-rc1
-$(package)_download_path=https://github.com/signal11/hidapi/archive
+$(package)_version=0.9.0
+$(package)_download_path=https://github.com/libusb/hidapi/archive
 $(package)_file_name=$(package)-$($(package)_version).tar.gz
-$(package)_sha256_hash=3c147200bf48a04c1e927cd81589c5ddceff61e6dac137a605f6ac9793f4af61
+$(package)_sha256_hash=630ee1834bdd5c5761ab079fd04f463a89585df8fcae51a7bfe4229b1e02a652
 $(package)_linux_dependencies=libusb eudev

 define $(package)_set_vars
@@ -28,3 +28,8 @@ endef
 define $(package)_stage_cmds
  $(MAKE) DESTDIR=$($(package)_staging_dir) install
 endef
+
+define $(package)_postprocess_cmds
+  rm lib/*.la
+endef
+
@@ -21,11 +21,6 @@ define $(package)_config_cmds
  $(MAKE) $($(package)_build_opts)
 endef

-#define $(package)_build_cmds
-#  cd source &&\
-   $(MAKE) $($((package)_build_opts) `nproc`
-#endef
-
 define $(package)_stage_cmds
  cd buildb &&\
  $(MAKE) $($(package)_build_opts) DESTDIR=$($(package)_staging_dir) install lib/*
@@ -6,8 +6,8 @@ $(package)_sha256_hash=8b88e059452118e8949a2752a55ce59bc71fa5bc414103e17f5b6b06f
 $(package)_dependencies=openssl

 define $(package)_set_vars
-  $(package)_config_opts=--disable-shared --enable-static --disable-dane-ta-usage --with-drill 
-  $(package)_config_opts=--with-ssl=$(host_prefix) 
+  $(package)_config_opts=--disable-shared --enable-static --with-drill
+  $(package)_config_opts+=--with-ssl=$(host_prefix)
  $(package)_config_opts_release=--disable-debug-mode
  $(package)_config_opts_linux=--with-pic
 endef
@@ -25,4 +25,6 @@ define $(package)_stage_cmds
 endef

 define $(package)_postprocess_cmds
+  rm lib/*.la
 endef
+
@@ -28,3 +28,7 @@ endef
 define $(package)_stage_cmds
  $(MAKE) DESTDIR=$($(package)_staging_dir) install
 endef
+
+define $(package)_postprocess_cmds
+  rm lib/*.la
+endef
@@ -0,0 +1,58 @@
+package=ncurses
+$(package)_version=6.1
+$(package)_download_path=https://ftp.gnu.org/gnu/ncurses
+$(package)_file_name=$(package)-$($(package)_version).tar.gz
+$(package)_sha256_hash=aa057eeeb4a14d470101eff4597d5833dcef5965331be3528c08d99cebaa0d17
+
+define $(package)_set_vars
+  $(package)_build_opts=CC="$($(package)_cc)"
+  $(package)_config_env=AR="$($(package)_ar)" RANLIB="$($(package)_ranlib)" CC="$($(package)_cc)"
+  $(package)_config_env_darwin=RANLIB="$(host_prefix)/native/bin/x86_64-apple-darwin11-ranlib" AR="$(host_prefix)/native/bin/x86_64-apple-darwin11-ar" CC="$(host_prefix)/native/bin/$($(package)_cc)"
+  $(package)_config_opts=--prefix=$(host_prefix)
+  $(package)_config_opts+=--disable-shared
+  $(package)_config_opts+=--with-build-cc=gcc
+  $(package)_config_opts+=--without-debug
+  $(package)_config_opts+=--without-ada
+  $(package)_config_opts+=--without-cxx-binding
+  $(package)_config_opts+=--without-cxx
+  $(package)_config_opts+=--without-ticlib
+  $(package)_config_opts+=--without-tic
+  $(package)_config_opts+=--without-progs
+  $(package)_config_opts+=--without-tests
+  $(package)_config_opts+=--without-tack
+  $(package)_config_opts+=--without-manpages
+  $(package)_config_opts+=--disable-tic-depends
+  $(package)_config_opts+=--disable-big-strings
+  $(package)_config_opts+=--disable-ext-colors
+  $(package)_config_opts+=--enable-pc-files
+  $(package)_config_opts+=--host=$(HOST)
+  $(pacakge)_config_opts+=--without-shared
+  $(pacakge)_config_opts+=--without-pthread
+  $(pacakge)_config_opts+=--disable-rpath
+  $(pacakge)_config_opts+=--disable-colorfgbg
+  $(pacakge)_config_opts+=--disable-ext-colors
+  $(pacakge)_config_opts+=--disable-ext-mouse
+  $(pacakge)_config_opts+=--disable-symlinks
+  $(pacakge)_config_opts+=--enable-warnings
+  $(pacakge)_config_opts+=--enable-assertions
+  $(pacakge)_config_opts+=--disable-home-terminfo
+  $(pacakge)_config_opts+=--enable-database
+  $(pacakge)_config_opts+=--enable-sp-funcs
+  $(pacakge)_config_opts+=--enable-term-driver
+  $(pacakge)_config_opts+=--enable-interop
+  $(pacakge)_config_opts+=--enable-widec
+  $(package)_build_opts=CFLAGS="$($(package)_cflags) $($(package)_cppflags) -fPIC"
+endef
+
+define $(package)_config_cmds
+  ./configure $($(package)_config_opts)
+endef
+
+define $(package)_build_cmds
+  $(MAKE) $($(package)_build_opts) V=1
+endef
+
+define $(package)_stage_cmds
+  $(MAKE) install DESTDIR=$($(package)_staging_dir)
+endef
+
@@ -1,8 +1,8 @@
 package=openssl
-$(package)_version=1.0.2q
+$(package)_version=1.0.2r
 $(package)_download_path=https://www.openssl.org/source
 $(package)_file_name=$(package)-$($(package)_version).tar.gz
-$(package)_sha256_hash=5744cfcbcec2b1b48629f7354203bc1e5e9b5466998bbccc5b5fcde3b18eb684
+$(package)_sha256_hash=ae51d08bba8a83958e894946f15303ff894d75c2b8bbd44a852b64e3fe11d0d6

 define $(package)_set_vars
 $(package)_config_env=AR="$($(package)_ar)" RANLIB="$($(package)_ranlib)" CC="$($(package)_cc)"
@@ -1,20 +1,16 @@
-packages:=boost openssl zeromq cppzmq expat ldns cppzmq readline libiconv hidapi protobuf libusb
+packages:=boost openssl zeromq cppzmq expat ldns libiconv hidapi protobuf libusb
 native_packages := native_ccache native_protobuf

 darwin_native_packages = native_biplist native_ds_store native_mac_alias
-darwin_packages = sodium-darwin
+darwin_packages = sodium-darwin ncurses readline 

-linux_packages = eudev
+linux_packages = eudev ncurses readline unwind sodium
 qt_packages = qt

 ifeq ($(build_tests),ON)
 packages += gtest
 endif

-ifeq ($(host_os),linux)
-packages += unwind
-packages += sodium
-endif
 ifeq ($(host_os),mingw32)
 packages += icu4c
 packages += sodium
@@ -25,5 +25,7 @@ define $(package)_stage_cmds
 endef

 define $(package)_postprocess_cmds
-  rm lib/libprotoc.a
+  rm lib/libprotoc.a &&\
+  rm lib/*.la
 endef
+
@@ -3,19 +3,19 @@ $(package)_version=8.0
 $(package)_download_path=https://ftp.gnu.org/gnu/readline
 $(package)_file_name=$(package)-$($(package)_version).tar.gz
 $(package)_sha256_hash=e339f51971478d369f8a053a330a190781acb9864cf4c541060f12078948e461
+$(package)_dependencies=ncurses

 define $(package)_set_vars
-  $(package)_build_opts=CC="$($(package)_cc)"
-  $(package)_config_env=AR="$($(package)_ar)" RANLIB="$($(package)_ranlib)" CC="$($(package)_cc)"
  $(package)_config_opts=--prefix=$(host_prefix)
-  $(package)_config_opts+=--disable-shared --enable-multibye --without-purify --without-curses
+  $(package)_config_opts+=--exec-prefix=$(host_prefix)
+  $(package)_config_opts+=--host=$(HOST)
+  $(package)_config_opts+=--disable-shared --with-curses
  $(package)_config_opts_release=--disable-debug-mode
+  $(package)_config_opts_darwin+=RANLIB="$(host_prefix)/native/bin/x86_64-apple-darwin11-ranlib" AR="$(host_prefix)/native/bin/x86_64-apple-darwin11-ar" CC="$(host_prefix)/native/bin/$($(package)_cc)"
  $(package)_build_opts=CFLAGS="$($(package)_cflags) $($(package)_cppflags) -fPIC"
 endef

 define $(package)_config_cmds
-  export bash_cv_have_mbstate_t=yes &&\
-  export bash_cv_wcwidth_broken=yes &&\
  ./configure $($(package)_config_opts)
 endef

@@ -24,6 +24,6 @@ define $(package)_build_cmds
 endef

 define $(package)_stage_cmds
-  $(MAKE) DESTDIR=$($(package)_staging_dir) install
+  $(MAKE) install DESTDIR=$($(package)_staging_dir) prefix=$(host_prefix) exec-prefix=$(host_prefix)
 endef

@@ -23,3 +23,8 @@ endef
 define $(package)_stage_cmds
  $(MAKE) DESTDIR=$($(package)_staging_dir) install
 endef
+
+define $(package)_postprocess_cmds
+  rm lib/*.la
+endef
+
@@ -23,3 +23,8 @@ endef
 define $(package)_stage_cmds
  $(MAKE) DESTDIR=$($(package)_staging_dir) install
 endef
+
+define $(package)_postprocess_cmds
+  rm lib/*.la
+endef
+
@@ -19,4 +19,6 @@ define $(package)_stage_cmds
 endef

 define $(package)_postprocess_cmds
+  rm lib/*.la
 endef
+
@@ -30,5 +30,7 @@ define $(package)_stage_cmds
 endef

 define $(package)_postprocess_cmds
-  rm -rf bin share
+  rm -rf bin share &&\
+  rm lib/*.la
 endef
+
@@ -8,6 +8,7 @@ OPTION(BUILD_TESTS "Build tests." OFF)

 SET(STATIC ON)
 SET(UNBOUND_STATIC ON)
+SET(ARCH "default")

 SET(BUILD_TESTS @build_tests@)
 SET(TREZOR_DEBUG @build_tests@)
@@ -20,6 +21,8 @@ SET(ENV{PKG_CONFIG_PATH} @prefix@/lib/pkgconfig)
 SET(LRELEASE_PATH @prefix@/native/bin CACHE FILEPATH "path to lrelease" FORCE)

 SET(Readline_ROOT_DIR @prefix@)
+SET(Readline_INCLUDE_DIR @prefix@/include)
+SET(Termcap_LIBRARY @prefix@/lib/libncurses.a)

 SET(LIBUNWIND_INCLUDE_DIR @prefix@/include)
 SET(LIBUNWIND_LIBRARIES @prefix@/lib/libunwind.a)
@@ -54,6 +54,9 @@
 #undef MONERO_DEFAULT_LOG_CATEGORY
 #define MONERO_DEFAULT_LOG_CATEGORY "net"

+#define AGGRESSIVE_TIMEOUT_THRESHOLD 120 // sockets
+#define NEW_CONNECTION_TIMEOUT_LOCAL 1200000 // 2 minutes
+#define NEW_CONNECTION_TIMEOUT_REMOTE 10000 // 10 seconds
 #define DEFAULT_TIMEOUT_MS_LOCAL 1800000 // 30 minutes
 #define DEFAULT_TIMEOUT_MS_REMOTE 300000 // 5 minutes
 #define TIMEOUT_EXTRA_MS_PER_BYTE 0.2
@@ -189,7 +192,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)

    m_protocol_handler.after_init_connection();

-    reset_timer(get_default_timeout(), false);
+    reset_timer(boost::posix_time::milliseconds(m_local ? NEW_CONNECTION_TIMEOUT_LOCAL : NEW_CONNECTION_TIMEOUT_REMOTE), false);

    // first read on the raw socket to detect SSL for the server
    buffer_ssl_init_fill = 0;
@@ -691,7 +694,7 @@ PRAGMA_WARNING_DISABLE_VS(4355)
  {
    unsigned count;
    try { count = host_count(m_host); } catch (...) { count = 0; }
-    const unsigned shift = std::min(std::max(count, 1u) - 1, 8u);
+    const unsigned shift = get_state().sock_count > AGGRESSIVE_TIMEOUT_THRESHOLD ? std::min(std::max(count, 1u) - 1, 8u) : 0;
    boost::posix_time::milliseconds timeout(0);
    if (m_local)
      timeout = boost::posix_time::milliseconds(DEFAULT_TIMEOUT_MS_LOCAL >> shift);
@@ -730,8 +733,6 @@ PRAGMA_WARNING_DISABLE_VS(4355)
  template<class t_protocol_handler>
  void connection<t_protocol_handler>::reset_timer(boost::posix_time::milliseconds ms, bool add)
  {
-    if (m_connection_type != e_connection_type_RPC)
-      return;
    MTRACE("Setting " << ms << " expiry");
    auto self = safe_shared_from_this();
    if(!self)
@@ -0,0 +1,38 @@
+// Copyright (c) 2019, The Monero Project
+//
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without modification, are
+// permitted provided that the following conditions are met:
+//
+// 1. Redistributions of source code must retain the above copyright notice, this list of
+//    conditions and the following disclaimer.
+//
+// 2. Redistributions in binary form must reproduce the above copyright notice, this list
+//    of conditions and the following disclaimer in the documentation and/or other
+//    materials provided with the distribution.
+//
+// 3. Neither the name of the copyright holder nor the names of its contributors may be
+//    used to endorse or promote products derived from this software without specific
+//    prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#pragma once
+
+namespace epee
+{
+  namespace net_utils
+  {
+    struct ssl_authentication_t;
+    class ssl_options_t;
+  }
+}
@@ -294,6 +294,11 @@ namespace net_utils
                               m_max_speed_up(0)
    {}

+    connection_context_base(const connection_context_base& a): connection_context_base()
+    {
+      set_details(a.m_connection_id, a.m_remote_address, a.m_is_income, a.m_ssl);
+    }
+
    connection_context_base& operator=(const connection_context_base& a)
    {
      set_details(a.m_connection_id, a.m_remote_address, a.m_is_income, a.m_ssl);
@@ -136,6 +136,7 @@ namespace epee
      //for pod types
      array_entry_t<type_name> sa;
      size_t size = read_varint();
+      CHECK_AND_ASSERT_THROW_MES(size <= m_count, "Size sanity check failed");
      sa.reserve(size);
      //TODO: add some optimization here later
      while(size--)
@@ -64,7 +64,8 @@ void buffer::append(const void *data, size_t sz)
      size_t reserve = (((size() + sz) * 3 / 2) + 4095) & ~4095;
      new_storage.reserve(reserve);
      new_storage.resize(size());
-      memcpy(new_storage.data(), storage.data() + offset, storage.size() - offset);
+      if (size() > 0)
+        memcpy(new_storage.data(), storage.data() + offset, storage.size() - offset);
      offset = 0;
      std::swap(storage, new_storage);
    }
@@ -62,13 +62,15 @@ wipeable_string::wipeable_string(wipeable_string &&other)
 wipeable_string::wipeable_string(const std::string &other)
 {
  grow(other.size());
-  memcpy(buffer.data(), other.c_str(), size());
+  if (size() > 0)
+    memcpy(buffer.data(), other.c_str(), size());
 }

 wipeable_string::wipeable_string(std::string &&other)
 {
  grow(other.size());
-  memcpy(buffer.data(), other.c_str(), size());
+  if (size() > 0)
+    memcpy(buffer.data(), other.c_str(), size());
  if (!other.empty())
  {
    memwipe(&other[0], other.size()); // we're kinda left with this again aren't we
@@ -79,7 +81,8 @@ wipeable_string::wipeable_string(std::string &&other)
 wipeable_string::wipeable_string(const char *s)
 {
  grow(strlen(s));
-  memcpy(buffer.data(), s, size());
+  if (size() > 0)
+    memcpy(buffer.data(), s, size());
 }

 wipeable_string::wipeable_string(const char *s, size_t len)
@@ -112,14 +115,18 @@ void wipeable_string::grow(size_t sz, size_t reserved)
  }
  size_t old_sz = buffer.size();
  std::unique_ptr<char[]> tmp{new char[old_sz]};
-  memcpy(tmp.get(), buffer.data(), old_sz * sizeof(char));
  if (old_sz > 0)
+  {
+    memcpy(tmp.get(), buffer.data(), old_sz * sizeof(char));
    memwipe(buffer.data(), old_sz * sizeof(char));
+  }
  buffer.reserve(reserved);
  buffer.resize(sz);
-  memcpy(buffer.data(), tmp.get(), old_sz * sizeof(char));
  if (old_sz > 0)
+  {
+    memcpy(buffer.data(), tmp.get(), old_sz * sizeof(char));
    memwipe(tmp.get(), old_sz * sizeof(char));
+  }
 }

 void wipeable_string::push_back(char c)
@@ -129,6 +129,7 @@ script: |
  chmod +x ${WRAP_DIR}/${prog}
  done

+  git config --global core.abbrev 9
  cd monero
  BASEPREFIX=`pwd`/contrib/depends
  # Build dependencies for each host
@@ -153,7 +154,7 @@ script: |
    export PATH=${BASEPREFIX}/${i}/native/bin:${ORIGPATH}
    mkdir build && cd build
    cmake .. -DCMAKE_TOOLCHAIN_FILE=${BASEPREFIX}/${i}/share/toolchain.cmake -DBACKCOMPAT=ON
-    make
+    make ${MAKEOPTS}
    DISTNAME=monero-${i}
    mv bin ${DISTNAME}
    find ${DISTNAME}/ | sort | tar --no-recursion --mode='u+rw,go+r-w,a+X' --owner=0 --group=0 -c -T - | gzip -9n > ${OUTDIR}/${DISTNAME}.tar.gz
@@ -77,6 +77,7 @@ script: |
  create_per-host_faketime_wrappers "2000-01-01 12:00:00"
  export PATH=${WRAP_DIR}:${PATH}

+  git config --global core.abbrev 9
  cd monero
  BASEPREFIX=`pwd`/contrib/depends

@@ -100,7 +101,7 @@ script: |
    export PATH=${BASEPREFIX}/${i}/native/bin:${ORIGPATH}
    mkdir build && cd build
    cmake .. -DCMAKE_TOOLCHAIN_FILE=${BASEPREFIX}/${i}/share/toolchain.cmake
-    make
+    make ${MAKEOPTS}
    DISTNAME=monero-${i}
    mv bin ${DISTNAME}
    find ${DISTNAME}/ | sort | tar --no-recursion --mode='u+rw,go+r-w,a+X' --owner=0 --group=0 -c -T - | gzip -9n > ${OUTDIR}/${DISTNAME}.tar.gz
@@ -100,6 +100,7 @@ script: |
  create_per-host_linker_wrapper "2000-01-01 12:00:00"
  export PATH=${WRAP_DIR}:${PATH}

+  git config --global core.abbrev 9
  cd monero
  BASEPREFIX=`pwd`/contrib/depends
  # Build dependencies for each host
@@ -125,7 +126,7 @@ script: |
    export PATH=${BASEPREFIX}/${i}/native/bin:${ORIGPATH}
    mkdir build && cd build
    cmake .. -DCMAKE_TOOLCHAIN_FILE=${BASEPREFIX}/${i}/share/toolchain.cmake
-    make
+    make ${MAKEOPTS}
    DISTNAME=monero-${i}
    mv bin ${DISTNAME}
    find ${DISTNAME}/ | sort | zip -X@ ${OUTDIR}/${DISTNAME}.zip
@@ -753,6 +753,21 @@ public:
   */
  virtual void batch_stop() = 0;

+  /**
+   * @brief aborts a batch transaction
+   *
+   * If the subclass implements batching, this function should abort the
+   * batch it is currently on.
+   *
+   * If no batch is in-progress, this function should throw a DB_ERROR.
+   * This exception may change in the future if it is deemed necessary to
+   * have a more granular exception type for this scenario.
+   *
+   * If any of this cannot be done, the subclass should throw the corresponding
+   * subclass of DB_EXCEPTION
+   */
+  virtual void batch_abort() = 0;
+
  /**
   * @brief sets whether or not to batch transactions
   *
@@ -1077,11 +1077,11 @@ void BlockchainLMDB::add_tx_amount_output_indices(const uint64_t tx_id,

  int result = 0;

-  int num_outputs = amount_output_indices.size();
+  size_t num_outputs = amount_output_indices.size();

  MDB_val_set(k_tx_id, tx_id);
  MDB_val v;
-  v.mv_data = (void *)amount_output_indices.data();
+  v.mv_data = num_outputs ? (void *)amount_output_indices.data() : (void*)"";
  v.mv_size = sizeof(uint64_t) * num_outputs;
  // LOG_PRINT_L1("tx_outputs[tx_hash] size: " << v.mv_size);

@@ -1953,7 +1953,7 @@ bool BlockchainLMDB::prune_worker(int mode, uint32_t pruning_seed)

  TIME_MEASURE_START(t);

-  size_t n_total_records = 0, n_prunable_records = 0, n_pruned_records = 0;
+  size_t n_total_records = 0, n_prunable_records = 0, n_pruned_records = 0, commit_counter = 0;
  uint64_t n_bytes = 0;

  mdb_txn_safe txn;
@@ -2056,6 +2056,7 @@ bool BlockchainLMDB::prune_worker(int mode, uint32_t pruning_seed)
          {
            MDEBUG("Pruning at height " << block_height << "/" << blockchain_height);
            ++n_pruned_records;
+            ++commit_counter;
            n_bytes += k.mv_size + v.mv_size;
            result = mdb_cursor_del(c_txs_prunable, 0);
            if (result)
@@ -2065,6 +2066,25 @@ bool BlockchainLMDB::prune_worker(int mode, uint32_t pruning_seed)
        result = mdb_cursor_del(c_txs_prunable_tip, 0);
        if (result)
          throw0(DB_ERROR(lmdb_error("Failed to delete transaction tip data: ", result).c_str()));
+
+        if (mode != prune_mode_check && commit_counter >= 4096)
+        {
+          MDEBUG("Committing txn at checkpoint...");
+          txn.commit();
+          result = mdb_txn_begin(m_env, NULL, 0, txn);
+          if (result)
+            throw0(DB_ERROR(lmdb_error("Failed to create a transaction for the db: ", result).c_str()));
+          result = mdb_cursor_open(txn, m_txs_pruned, &c_txs_pruned);
+          if (result)
+            throw0(DB_ERROR(lmdb_error("Failed to open a cursor for txs_pruned: ", result).c_str()));
+          result = mdb_cursor_open(txn, m_txs_prunable, &c_txs_prunable);
+          if (result)
+            throw0(DB_ERROR(lmdb_error("Failed to open a cursor for txs_prunable: ", result).c_str()));
+          result = mdb_cursor_open(txn, m_txs_prunable_tip, &c_txs_prunable_tip);
+          if (result)
+            throw0(DB_ERROR(lmdb_error("Failed to open a cursor for txs_prunable_tip: ", result).c_str()));
+          commit_counter = 0;
+        }
      }
    }
  }
@@ -2134,6 +2154,7 @@ bool BlockchainLMDB::prune_worker(int mode, uint32_t pruning_seed)
            result = mdb_cursor_del(c_txs_prunable, 0);
            if (result)
              throw0(DB_ERROR(lmdb_error("Failed to delete transaction prunable data: ", result).c_str()));
+            ++commit_counter;
          }
        }
      }
@@ -2150,6 +2171,34 @@ bool BlockchainLMDB::prune_worker(int mode, uint32_t pruning_seed)
                ", seed " << epee::string_tools::to_string_hex(pruning_seed));
        }
      }
+
+      if (mode != prune_mode_check && commit_counter >= 4096)
+      {
+        MDEBUG("Committing txn at checkpoint...");
+        txn.commit();
+        result = mdb_txn_begin(m_env, NULL, 0, txn);
+        if (result)
+          throw0(DB_ERROR(lmdb_error("Failed to create a transaction for the db: ", result).c_str()));
+        result = mdb_cursor_open(txn, m_txs_pruned, &c_txs_pruned);
+        if (result)
+          throw0(DB_ERROR(lmdb_error("Failed to open a cursor for txs_pruned: ", result).c_str()));
+        result = mdb_cursor_open(txn, m_txs_prunable, &c_txs_prunable);
+        if (result)
+          throw0(DB_ERROR(lmdb_error("Failed to open a cursor for txs_prunable: ", result).c_str()));
+        result = mdb_cursor_open(txn, m_txs_prunable_tip, &c_txs_prunable_tip);
+        if (result)
+          throw0(DB_ERROR(lmdb_error("Failed to open a cursor for txs_prunable_tip: ", result).c_str()));
+        result = mdb_cursor_open(txn, m_tx_indices, &c_tx_indices);
+        if (result)
+          throw0(DB_ERROR(lmdb_error("Failed to open a cursor for tx_indices: ", result).c_str()));
+        MDB_val val;
+        val.mv_size = sizeof(ti);
+        val.mv_data = (void *)&ti;
+        result = mdb_cursor_get(c_tx_indices, (MDB_val*)&zerokval, &val, MDB_GET_BOTH);
+        if (result)
+          throw0(DB_ERROR(lmdb_error("Failed to restore cursor for tx_indices: ", result).c_str()));
+        commit_counter = 0;
+      }
    }
    mdb_cursor_close(c_tx_indices);
  }
@@ -54,6 +54,7 @@ public:
  virtual void unlock() override { }
  virtual bool batch_start(uint64_t batch_num_blocks=0, uint64_t batch_bytes=0) override { return true; }
  virtual void batch_stop() override {}
+  virtual void batch_abort() override {}
  virtual void set_batch_transactions(bool) override {}
  virtual void block_wtxn_start() override {}
  virtual void block_wtxn_stop() override {}
@@ -209,6 +209,7 @@ namespace cryptonote
    ADD_CHECKPOINT(1579000, "7d0d7a2346373afd41ed1e744a939fc5d474a7dbaa257be5c6fff4009e789241");
    ADD_CHECKPOINT(1668900, "ac2dcaf3d2f58ffcf8391639f0f1ebafcb8eac43c49479c7c37f611868d07568");
    ADD_CHECKPOINT(1775600, "1c6e01c661dc22cab939e79ec6a5272190624ce8356d2f7b958e4f9a57fdb05e");
+    ADD_CHECKPOINT(1856000, "9b57f17f29c71a3acd8a7904b93c41fa6eb8d2b7c73936ce4f1702d14880ba29");
    return true;
  }

@@ -48,7 +48,6 @@ static const char *DEFAULT_DNS_PUBLIC_ADDR[] =
  "80.67.169.40",       // FDN (France)
  "89.233.43.71",       // http://censurfridns.dk (Denmark)
  "109.69.8.51",        // punCAT (Spain)
-  "77.109.148.137",     // Xiala.net (Switzerland)
  "193.58.251.251",     // SkyDNS (Russia)
 };

@@ -36,6 +36,7 @@
 #include "storages/http_abstract_invoke.h"
 #include "net/http_auth.h"
 #include "net/http_client.h"
+#include "net/net_ssl.h"
 #include "string_tools.h"

 namespace tools
@@ -49,11 +50,12 @@ namespace tools
        uint32_t ip
      , uint16_t port
      , boost::optional<epee::net_utils::http::login> user
+      , epee::net_utils::ssl_options_t ssl_options
      )
      : m_http_client{}
    {
      m_http_client.set_server(
-        epee::string_tools::get_ip_string_from_int32(ip), std::to_string(port), std::move(user)
+        epee::string_tools::get_ip_string_from_int32(ip), std::to_string(port), std::move(user), std::move(ssl_options)
      );
    }

@@ -105,9 +105,12 @@ void keccak(const uint8_t *in, size_t inlen, uint8_t *md, int mdlen)
    memset(st, 0, sizeof(st));

    for ( ; inlen >= rsiz; inlen -= rsiz, in += rsiz) {
-        for (i = 0; i < rsizw; i++)
-            st[i] ^= swap64le(((uint64_t *) in)[i]);
-        keccakf(st, KECCAK_ROUNDS);
+      for (i = 0; i < rsizw; i++) {
+        uint64_t ina;
+        memcpy(&ina, in + i * 8, 8);
+        st[i] ^= swap64le(ina);
+      }
+      keccakf(st, KECCAK_ROUNDS);
    }
    
    // last block and padding
@@ -116,7 +119,8 @@ void keccak(const uint8_t *in, size_t inlen, uint8_t *md, int mdlen)
      local_abort("Bad keccak use");
    }

-    memcpy(temp, in, inlen);
+    if (inlen > 0)
+      memcpy(temp, in, inlen);
    temp[inlen++] = 1;
    memset(temp + inlen, 0, rsiz - inlen);
    temp[rsiz - 1] |= 0x80;
@@ -30,6 +30,7 @@

 #include <assert.h>
 #include <stddef.h>
+#include <stdlib.h>
 #include <string.h>

 #include "hash-ops.h"
@@ -82,23 +83,24 @@ void tree_hash(const char (*hashes)[HASH_SIZE], size_t count, char *root_hash) {

    size_t cnt = tree_hash_cnt( count );

-    char ints[cnt][HASH_SIZE];
-    memset(ints, 0 , sizeof(ints));  // zero out as extra protection for using uninitialized mem
+    char *ints = calloc(cnt, HASH_SIZE);  // zero out as extra protection for using uninitialized mem
+    assert(ints);

    memcpy(ints, hashes, (2 * cnt - count) * HASH_SIZE);

    for (i = 2 * cnt - count, j = 2 * cnt - count; j < cnt; i += 2, ++j) {
-      cn_fast_hash(hashes[i], 64, ints[j]);
+      cn_fast_hash(hashes[i], 64, ints + j * HASH_SIZE);
    }
    assert(i == count);

    while (cnt > 2) {
      cnt >>= 1;
      for (i = 0, j = 0; j < cnt; i += 2, ++j) {
-        cn_fast_hash(ints[i], 64, ints[j]);
+        cn_fast_hash(ints + i * HASH_SIZE, 64, ints + j * HASH_SIZE);
      }
    }

-    cn_fast_hash(ints[0], 64, root_hash);
+    cn_fast_hash(ints, 64, root_hash);
+    free(ints);
  }
 }
@@ -320,7 +320,7 @@ namespace cryptonote
      }
      if (!typename Archive<W>::is_saving())
        pruned = true;
-      return true;
+      return ar.stream().good();
    }

  private:
@@ -221,8 +221,7 @@ namespace cryptonote
    tx.invalidate_hashes();
    //TODO: validate tx

-    get_transaction_hash(tx, tx_hash);
-    return true;
+    return get_transaction_hash(tx, tx_hash);
  }
  //---------------------------------------------------------------
  bool parse_and_validate_tx_from_blob(const blobdata& tx_blob, transaction& tx, crypto::hash& tx_hash, crypto::hash& tx_prefix_hash)
@@ -975,6 +974,7 @@ namespace cryptonote
  {
    crypto::hash h = null_hash;
    get_transaction_hash(t, h, NULL);
+    CHECK_AND_ASSERT_THROW_MES(get_transaction_hash(t, h, NULL), "Failed to calculate transaction hash");
    return h;
  }
  //---------------------------------------------------------------
@@ -1327,7 +1327,7 @@ namespace cryptonote
    txs_ids.reserve(1 + b.tx_hashes.size());
    crypto::hash h = null_hash;
    size_t bl_sz = 0;
-    get_transaction_hash(b.miner_tx, h, bl_sz);
+    CHECK_AND_ASSERT_THROW_MES(get_transaction_hash(b.miner_tx, h, bl_sz), "Failed to calculate transaction hash");
    txs_ids.push_back(h);
    for(auto& th: b.tx_hashes)
      txs_ids.push_back(th);
@@ -123,7 +123,7 @@ namespace cryptonote
    m_miner_extra_sleep(BACKGROUND_MINING_DEFAULT_MINER_EXTRA_SLEEP_MILLIS),
    m_block_reward(0)
  {
-
+    m_attrs.set_stack_size(THREAD_STACK_SIZE);
  }
  //-----------------------------------------------------------------------------------------------------
  miner::~miner()
@@ -360,7 +360,7 @@ namespace cryptonote
    return m_threads_total;
  }
  //-----------------------------------------------------------------------------------------------------
-  bool miner::start(const account_public_address& adr, size_t threads_count, const boost::thread::attributes& attrs, bool do_background, bool ignore_battery)
+  bool miner::start(const account_public_address& adr, size_t threads_count, bool do_background, bool ignore_battery)
  {
    m_block_reward = 0;
    m_mine_address = adr;
@@ -371,7 +371,6 @@ namespace cryptonote
      m_threads_autodetect.push_back({epee::misc_utils::get_ns_count(), m_total_hashes});
      m_threads_total = 1;
    }
-    m_attrs = attrs;
    m_starter_nonce = crypto::rand<uint32_t>();
    CRITICAL_REGION_LOCAL(m_threads_lock);
    if(is_mining())
@@ -395,7 +394,7 @@ namespace cryptonote
    
    for(size_t i = 0; i != m_threads_total; i++)
    {
-      m_threads.push_back(boost::thread(attrs, boost::bind(&miner::worker_thread, this)));
+      m_threads.push_back(boost::thread(m_attrs, boost::bind(&miner::worker_thread, this)));
    }

    if (threads_count == 0)
@@ -405,7 +404,7 @@ namespace cryptonote

    if( get_is_background_mining_enabled() )
    {
-      m_background_mining_thread = boost::thread(attrs, boost::bind(&miner::background_worker_thread, this));
+      m_background_mining_thread = boost::thread(m_attrs, boost::bind(&miner::background_worker_thread, this));
      LOG_PRINT_L0("Background mining controller thread started" );
    }

@@ -487,10 +486,7 @@ namespace cryptonote
  {
    if(m_do_mining)
    {
-      boost::thread::attributes attrs;
-      attrs.set_stack_size(THREAD_STACK_SIZE);
-
-      start(m_mine_address, m_threads_total, attrs, get_is_background_mining_enabled(), get_ignore_battery());
+      start(m_mine_address, m_threads_total, get_is_background_mining_enabled(), get_ignore_battery());
    }
  }
  //-----------------------------------------------------------------------------------------------------
@@ -64,7 +64,7 @@ namespace cryptonote
    static void init_options(boost::program_options::options_description& desc);
    bool set_block_template(const block& bl, const difficulty_type& diffic, uint64_t height, uint64_t block_reward);
    bool on_block_chain_update();
-    bool start(const account_public_address& adr, size_t threads_count, const boost::thread::attributes& attrs, bool do_background = false, bool ignore_battery = false);
+    bool start(const account_public_address& adr, size_t threads_count, bool do_background = false, bool ignore_battery = false);
    uint64_t get_speed() const;
    uint32_t get_threads_count() const;
    void send_stop_signal();
@@ -182,7 +182,8 @@ Blockchain::Blockchain(tx_memory_pool& tx_pool) :
  m_long_term_block_weights_cache_rolling_median(CRYPTONOTE_LONG_TERM_BLOCK_WEIGHT_WINDOW_SIZE),
  m_difficulty_for_next_block_top_hash(crypto::null_hash),
  m_difficulty_for_next_block(1),
-  m_btc_valid(false)
+  m_btc_valid(false),
+  m_batch_success(true)
 {
  LOG_PRINT_L3("Blockchain::" << __func__);
 }
@@ -619,17 +620,13 @@ void Blockchain::pop_blocks(uint64_t nblocks)
  CRITICAL_REGION_LOCAL(m_tx_pool);
  CRITICAL_REGION_LOCAL1(m_blockchain_lock);

-  while (!m_db->batch_start())
-  {
-    m_blockchain_lock.unlock();
-    m_tx_pool.unlock();
-    epee::misc_utils::sleep_no_w(1000);
-    m_tx_pool.lock();
-    m_blockchain_lock.lock();
-  }
+  bool stop_batch = m_db->batch_start();

  try
  {
+    const uint64_t blockchain_height = m_db->height();
+    if (blockchain_height > 0)
+      nblocks = std::min(nblocks, blockchain_height - 1);
    for (i=0; i < nblocks; ++i)
    {
      pop_block_from_blockchain();
@@ -637,10 +634,14 @@ void Blockchain::pop_blocks(uint64_t nblocks)
  }
  catch (const std::exception& e)
  {
-    LOG_ERROR("Error when popping blocks, only " << i << " blocks are popped: " << e.what());
+    LOG_ERROR("Error when popping blocks after processing " << i << " blocks: " << e.what());
+    if (stop_batch)
+      m_db->batch_abort();
+    return;
  }

-  m_db->batch_stop();
+  if (stop_batch)
+    m_db->batch_stop();
 }
 //------------------------------------------------------------------
 // This function tells BlockchainDB to remove the top block from the
@@ -1373,7 +1374,8 @@ bool Blockchain::create_block_template(block& b, const crypto::hash *from_block,
    // just as we compare it, we'll just use a slightly old template, but
    // this would be the case anyway if we'd lock, and the change happened
    // just after the block template was created
-    if (!memcmp(&miner_address, &m_btc_address, sizeof(cryptonote::account_public_address)) && m_btc_nonce == ex_nonce && m_btc_pool_cookie == m_tx_pool.cookie()) {
+    if (!memcmp(&miner_address, &m_btc_address, sizeof(cryptonote::account_public_address)) && m_btc_nonce == ex_nonce
+      && m_btc_pool_cookie == m_tx_pool.cookie() && m_btc.prev_id == get_tail_id()) {
      MDEBUG("Using cached template");
      m_btc.timestamp = time(NULL); // update timestamp unconditionally
      b = m_btc;
@@ -1891,10 +1893,14 @@ bool Blockchain::handle_get_objects(NOTIFY_REQUEST_GET_OBJECTS::request& arg, NO

    if (missed_tx_ids.size() != 0)
    {
-      LOG_ERROR("Error retrieving blocks, missed " << missed_tx_ids.size()
-          << " transactions for block with hash: " << get_block_hash(bl.second)
-          << std::endl
-      );
+      // do not display an error if the peer asked for an unpruned block which we are not meant to have
+      if (tools::has_unpruned_block(get_block_height(bl.second), get_current_blockchain_height(), get_blockchain_pruning_seed()))
+      {
+        LOG_ERROR("Error retrieving blocks, missed " << missed_tx_ids.size()
+            << " transactions for block with hash: " << get_block_hash(bl.second)
+            << std::endl
+        );
+      }

      // append missed transaction hashes to response missed_ids field,
      // as done below if any standalone transactions were requested
@@ -3844,6 +3850,7 @@ leave:
    catch (const KEY_IMAGE_EXISTS& e)
    {
      LOG_ERROR("Error adding block with hash: " << id << " to blockchain, what = " << e.what());
+      m_batch_success = false;
      bvc.m_verifivation_failed = true;
      return_tx_to_pool(txs);
      return false;
@@ -3852,6 +3859,7 @@ leave:
    {
      //TODO: figure out the best way to deal with this failure
      LOG_ERROR("Error adding block with hash: " << id << " to blockchain, what = " << e.what());
+      m_batch_success = false;
      bvc.m_verifivation_failed = true;
      return_tx_to_pool(txs);
      return false;
@@ -4161,7 +4169,10 @@ bool Blockchain::cleanup_handle_incoming_blocks(bool force_sync)

  try
  {
-    m_db->batch_stop();
+    if (m_batch_success)
+      m_db->batch_stop();
+    else
+      m_db->batch_abort();
    success = true;
  }
  catch (const std::exception &e)
@@ -4385,6 +4396,7 @@ bool Blockchain::prepare_handle_incoming_blocks(const std::vector<block_complete
    m_tx_pool.lock();
    m_blockchain_lock.lock();
  }
+  m_batch_success = true;

  const uint64_t height = m_db->height();
  if ((height + blocks_entry.size()) < m_blocks_hash_check.size())
@@ -4837,7 +4849,7 @@ void Blockchain::cancel()
 }

 #if defined(PER_BLOCK_CHECKPOINT)
-static const char expected_block_hashes_hash[] = "570ce2357b08fadac6058e34f95c5e08323f9325de260d07b091a281a948a7b0";
+static const char expected_block_hashes_hash[] = "7dafb40b414a0e59bfced6682ef519f0b416bc914dd3d622b72e0dd1a47117c2";
 void Blockchain::load_compiled_in_block_hashes(const GetCheckpointsCallback& get_checkpoints)
 {
  if (get_checkpoints == nullptr || !m_fast_sync)
@@ -1102,6 +1102,9 @@ namespace cryptonote
    uint64_t m_btc_expected_reward;
    bool m_btc_valid;

+
+    bool m_batch_success;
+
    std::shared_ptr<tools::Notify> m_block_notify;
    std::shared_ptr<tools::Notify> m_reorg_notify;

@@ -674,9 +674,15 @@ namespace cryptonote
    if (prune_blockchain)
    {
      // display a message if the blockchain is not pruned yet
-      if (m_blockchain_storage.get_current_blockchain_height() > 1 && !m_blockchain_storage.get_blockchain_pruning_seed())
+      if (!m_blockchain_storage.get_blockchain_pruning_seed())
+      {
        MGINFO("Pruning blockchain...");
-      CHECK_AND_ASSERT_MES(m_blockchain_storage.prune_blockchain(), false, "Failed to prune blockchain");
+        CHECK_AND_ASSERT_MES(m_blockchain_storage.prune_blockchain(), false, "Failed to prune blockchain");
+      }
+      else
+      {
+        CHECK_AND_ASSERT_MES(m_blockchain_storage.update_blockchain_pruning(), false, "Failed to update blockchain pruning");
+      }
    }

    return load_state_data();
@@ -95,13 +95,17 @@ namespace cryptonote
    // the whole prepare/handle/cleanup incoming block sequence.
    class LockedTXN {
    public:
-      LockedTXN(Blockchain &b): m_blockchain(b), m_batch(false) {
+      LockedTXN(Blockchain &b): m_blockchain(b), m_batch(false), m_active(false) {
        m_batch = m_blockchain.get_db().batch_start();
+        m_active = true;
      }
-      ~LockedTXN() { try { if (m_batch) { m_blockchain.get_db().batch_stop(); } } catch (const std::exception &e) { MWARNING("LockedTXN dtor filtering exception: " << e.what()); } }
+      void commit() { try { if (m_batch && m_active) { m_blockchain.get_db().batch_stop(); m_active = false; } } catch (const std::exception &e) { MWARNING("LockedTXN::commit filtering exception: " << e.what()); } }
+      void abort() { try { if (m_batch && m_active) { m_blockchain.get_db().batch_abort(); m_active = false; } } catch (const std::exception &e) { MWARNING("LockedTXN::abort filtering exception: " << e.what()); } }
+      ~LockedTXN() { abort(); }
    private:
      Blockchain &m_blockchain;
      bool m_batch;
+      bool m_active;
    };
  }
  //---------------------------------------------------------------------------------
@@ -255,6 +259,7 @@ namespace cryptonote
          if (!insert_key_images(tx, id, kept_by_block))
            return false;
          m_txs_by_fee_and_receive_time.emplace(std::pair<double, std::time_t>(fee / (double)tx_weight, receive_time), id);
+          lock.commit();
        }
        catch (const std::exception &e)
        {
@@ -299,6 +304,7 @@ namespace cryptonote
        if (!insert_key_images(tx, id, kept_by_block))
          return false;
        m_txs_by_fee_and_receive_time.emplace(std::pair<double, std::time_t>(fee / (double)tx_weight, receive_time), id);
+        lock.commit();
      }
      catch (const std::exception &e)
      {
@@ -398,6 +404,7 @@ namespace cryptonote
        return;
      }
    }
+    lock.commit();
    if (changed)
      ++m_cookie;
    if (m_txpool_weight > bytes)
@@ -494,6 +501,7 @@ namespace cryptonote
      m_blockchain.remove_txpool_tx(id);
      m_txpool_weight -= tx_weight;
      remove_transaction_keyimages(tx, id);
+      lock.commit();
    }
    catch (const std::exception &e)
    {
@@ -578,6 +586,7 @@ namespace cryptonote
          // ignore error
        }
      }
+      lock.commit();
      ++m_cookie;
    }
    return true;
@@ -641,6 +650,7 @@ namespace cryptonote
        // continue
      }
    }
+    lock.commit();
  }
  //---------------------------------------------------------------------------------
  size_t tx_memory_pool::get_transactions_count(bool include_unrelayed_txes) const
@@ -1119,6 +1129,7 @@ namespace cryptonote
        }
      }
    }
+    lock.commit();
    if (changed)
      ++m_cookie;
  }
@@ -1271,6 +1282,7 @@ namespace cryptonote
      append_key_images(k_images, tx);
      LOG_PRINT_L2("  added, new block weight " << total_weight << "/" << max_total_weight << ", coinbase " << print_money(best_coinbase));
    }
+    lock.commit();

    expected_reward = best_coinbase;
    LOG_PRINT_L2("Block template filled with " << bl.tx_hashes.size() << " txes, weight "
@@ -1336,6 +1348,7 @@ namespace cryptonote
          // continue
        }
      }
+      lock.commit();
    }
    if (n_removed > 0)
      ++m_cookie;
@@ -1395,6 +1408,7 @@ namespace cryptonote
          // ignore error
        }
      }
+      lock.commit();
    }

    m_cookie = 0;
@@ -88,7 +88,7 @@ bool tx_sanity_check(Blockchain &blockchain, const cryptonote::blobdata &tx_blob

  std::vector<uint64_t> offsets(rct_indices.begin(), rct_indices.end());
  uint64_t median = epee::misc_utils::median(offsets);
-  if (median < n_available * 9 / 10)
+  if (median < n_available * 6 / 10)
  {
    MERROR("median is " << median << "/" << n_available);
    return false;
@@ -52,6 +52,7 @@ PUSH_WARNINGS
 DISABLE_VS_WARNINGS(4355)

 #define LOCALHOST_INT 2130706433
+#define CURRENCY_PROTOCOL_MAX_OBJECT_REQUEST_COUNT 500

 namespace cryptonote
 {
@@ -341,6 +341,11 @@ namespace cryptonote

    if(m_core.have_block(hshd.top_id))
    {
+      if (target > hshd.current_height)
+      {
+        MINFO(context << "peer is not ahead of us and we're syncing, disconnecting");
+        return false;
+      }
      context.m_state = cryptonote_connection_context::state_normal;
      if(is_inital && target == m_core.get_current_blockchain_height())
        on_connection_synchronized();
@@ -809,12 +814,27 @@ namespace cryptonote
    NOTIFY_NEW_FLUFFY_BLOCK::request fluffy_response;
    fluffy_response.b.block = t_serializable_object_to_blob(b);
    fluffy_response.current_blockchain_height = arg.current_blockchain_height;
+    std::vector<bool> seen(b.tx_hashes.size(), false);
    for(auto& tx_idx: arg.missing_tx_indices)
    {
      if(tx_idx < b.tx_hashes.size())
      {
        MDEBUG("  tx " << b.tx_hashes[tx_idx]);
+        if (seen[tx_idx])
+        {
+          LOG_ERROR_CCONTEXT
+          (
+            "Failed to handle request NOTIFY_REQUEST_FLUFFY_MISSING_TX"
+            << ", request is asking for duplicate tx "
+            << ", tx index = " << tx_idx << ", block tx count " << b.tx_hashes.size()
+            << ", block_height = " << arg.current_blockchain_height
+            << ", dropping connection"
+          );
+          drop_connection(context, true, false);
+          return 1;
+        }
        txids.push_back(b.tx_hashes[tx_idx]);
+        seen[tx_idx] = true;
      }
      else
      {
@@ -914,6 +934,17 @@ namespace cryptonote
  int t_cryptonote_protocol_handler<t_core>::handle_request_get_objects(int command, NOTIFY_REQUEST_GET_OBJECTS::request& arg, cryptonote_connection_context& context)
  {
    MLOG_P2P_MESSAGE("Received NOTIFY_REQUEST_GET_OBJECTS (" << arg.blocks.size() << " blocks, " << arg.txs.size() << " txes)");
+
+    if (arg.blocks.size() + arg.txs.size() > CURRENCY_PROTOCOL_MAX_OBJECT_REQUEST_COUNT)
+      {
+        LOG_ERROR_CCONTEXT(
+            "Requested objects count is too big ("
+            << arg.blocks.size() + arg.txs.size() << ") expected not more then "
+            << CURRENCY_PROTOCOL_MAX_OBJECT_REQUEST_COUNT);
+        drop_connection(context, false, false);
+        return 1;
+      }
+
    NOTIFY_RESPONSE_GET_OBJECTS::request rsp;
    if(!m_core.handle_get_objects(arg, rsp, context))
    {
@@ -40,10 +40,11 @@ t_command_parser_executor::t_command_parser_executor(
    uint32_t ip
  , uint16_t port
  , const boost::optional<tools::login>& login
+  , const epee::net_utils::ssl_options_t& ssl_options
  , bool is_rpc
  , cryptonote::core_rpc_server* rpc_server
  )
-  : m_executor(ip, port, login, is_rpc, rpc_server)
+  : m_executor(ip, port, login, ssl_options, is_rpc, rpc_server)
 {}

 bool t_command_parser_executor::print_peer_list(const std::vector<std::string>& args)
@@ -40,6 +40,7 @@

 #include "daemon/rpc_command_executor.h"
 #include "common/common_fwd.h"
+#include "net/net_fwd.h"
 #include "rpc/core_rpc_server.h"

 namespace daemonize {
@@ -53,6 +54,7 @@ public:
      uint32_t ip
    , uint16_t port
    , const boost::optional<tools::login>& login
+    , const epee::net_utils::ssl_options_t& ssl_options
    , bool is_rpc
    , cryptonote::core_rpc_server* rpc_server = NULL
    );
@@ -43,10 +43,11 @@ t_command_server::t_command_server(
    uint32_t ip
  , uint16_t port
  , const boost::optional<tools::login>& login
+  , const epee::net_utils::ssl_options_t& ssl_options
  , bool is_rpc
  , cryptonote::core_rpc_server* rpc_server
  )
-  : m_parser(ip, port, login, is_rpc, rpc_server)
+  : m_parser(ip, port, login, ssl_options, is_rpc, rpc_server)
  , m_command_lookup()
  , m_is_rpc(is_rpc)
 {
@@ -43,6 +43,7 @@ Passing RPC commands:
 #include "common/common_fwd.h"
 #include "console_handler.h"
 #include "daemon/command_parser_executor.h"
+#include "net/net_fwd.h"

 namespace daemonize {

@@ -57,6 +58,7 @@ public:
      uint32_t ip
    , uint16_t port
    , const boost::optional<tools::login>& login
+    , const epee::net_utils::ssl_options_t& ssl_options
    , bool is_rpc = true
    , cryptonote::core_rpc_server* rpc_server = NULL
    );
@@ -45,6 +45,7 @@
 #include "daemon/command_server.h"
 #include "daemon/command_server.h"
 #include "daemon/command_line_args.h"
+#include "net/net_ssl.h"
 #include "version.h"

 using namespace epee;
@@ -163,7 +164,7 @@ bool t_daemon::run(bool interactive)
    if (interactive && mp_internals->rpcs.size())
    {
      // The first three variables are not used when the fourth is false
-      rpc_commands.reset(new daemonize::t_command_server(0, 0, boost::none, false, mp_internals->rpcs.front()->get_server()));
+      rpc_commands.reset(new daemonize::t_command_server(0, 0, boost::none, epee::net_utils::ssl_support_t::e_ssl_support_disabled, false, mp_internals->rpcs.front()->get_server()));
      rpc_commands->start_handling(std::bind(&daemonize::t_daemon::stop_p2p, this));
    }

@@ -324,7 +324,11 @@ int main(int argc, char const * argv[])
          }
        }

-        daemonize::t_command_server rpc_commands{rpc_ip, rpc_port, std::move(login)};
+        auto ssl_options = cryptonote::rpc_args::process_ssl(vm, true);
+        if (!ssl_options)
+          return 1;
+
+        daemonize::t_command_server rpc_commands{rpc_ip, rpc_port, std::move(login), std::move(*ssl_options)};
        if (rpc_commands.process_command_vec(command))
        {
          return 0;
@@ -127,6 +127,7 @@ t_rpc_command_executor::t_rpc_command_executor(
    uint32_t ip
  , uint16_t port
  , const boost::optional<tools::login>& login
+  , const epee::net_utils::ssl_options_t& ssl_options
  , bool is_rpc
  , cryptonote::core_rpc_server* rpc_server
  )
@@ -137,7 +138,7 @@ t_rpc_command_executor::t_rpc_command_executor(
    boost::optional<epee::net_utils::http::login> http_login{};
    if (login)
      http_login.emplace(login->username, login->password.password());
-    m_rpc_client = new tools::t_rpc_client(ip, port, std::move(http_login));
+    m_rpc_client = new tools::t_rpc_client(ip, port, std::move(http_login), ssl_options);
  }
  else
  {
@@ -43,6 +43,7 @@
 #include "common/common_fwd.h"
 #include "common/rpc_client.h"
 #include "cryptonote_basic/cryptonote_basic.h"
+#include "net/net_fwd.h"
 #include "rpc/core_rpc_server.h"

 #undef MONERO_DEFAULT_LOG_CATEGORY
@@ -61,6 +62,7 @@ public:
      uint32_t ip
    , uint16_t port
    , const boost::optional<tools::login>& user
+    , const epee::net_utils::ssl_options_t& ssl_options
    , bool is_rpc = true
    , cryptonote::core_rpc_server* rpc_server = NULL
    );
@@ -90,6 +90,20 @@ namespace hw {
      AKout = keys.AKout;
    }

+    ABPkeys &ABPkeys::operator=(const ABPkeys& keys) {
+      if (&keys == this)
+        return *this;
+      Aout = keys.Aout;
+      Bout = keys.Bout;
+      is_subaddress = keys.is_subaddress;
+      is_change_address = keys.is_change_address;
+      additional_key = keys.additional_key;
+      index = keys.index;
+      Pout = keys.Pout;
+      AKout = keys.AKout;
+      return *this;
+    }
+
    bool Keymap::find(const rct::key& P, ABPkeys& keys) const {
      size_t sz = ABP.size();
      for (size_t i=0; i<sz; i++) {
@@ -77,6 +77,7 @@ namespace hw {
        ABPkeys(const rct::key& A, const rct::key& B, const bool is_subaddr, bool is_subaddress, bool is_change_address, size_t index, const rct::key& P,const rct::key& AK);
        ABPkeys(const ABPkeys& keys) ;
        ABPkeys() {index=0;is_subaddress=false;is_subaddress=false;is_change_address=false;}
+        ABPkeys &operator=(const ABPkeys &keys);
    };

    class Keymap {
@@ -944,7 +944,10 @@ namespace nodetool
      }
      if(!context.m_is_income)
        m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.set_peer_just_seen(context.peer_id, context.m_remote_address, context.m_pruning_seed, context.m_rpc_port);
-      m_payload_handler.process_payload_sync_data(rsp.payload_data, context, false);
+      if (!m_payload_handler.process_payload_sync_data(rsp.payload_data, context, false))
+      {
+        m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().close(context.m_connection_id );
+      }
    });

    if(!r)
@@ -1090,6 +1093,7 @@ namespace nodetool
      LOG_PRINT_CC_PRIORITY_NODE(is_priority, *con, "Failed to HANDSHAKE with peer "
        << na.str()
        /*<< ", try " << try_count*/);
+      zone.m_net_server.get_config_object().close(con->m_connection_id);
      return false;
    }

@@ -1149,7 +1153,7 @@ namespace nodetool
      bool is_priority = is_priority_node(na);

      LOG_PRINT_CC_PRIORITY_NODE(is_priority, *con, "Failed to HANDSHAKE with peer " << na.str());
-
+      zone.m_net_server.get_config_object().close(con->m_connection_id);
      return false;
    }

@@ -1226,19 +1230,53 @@ namespace nodetool
      size_t random_index;
      const uint32_t next_needed_pruning_stripe = m_payload_handler.get_next_needed_pruning_stripe().second;

+      // build a set of all the /16 we're connected to, and prefer a peer that's not in that set
+      std::set<uint32_t> classB;
+      if (&zone == &m_network_zones.at(epee::net_utils::zone::public_)) // at returns reference, not copy
+      {
+        zone.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt)
+        {
+          if (cntxt.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id())
+          {
+
+            const epee::net_utils::network_address na = cntxt.m_remote_address;
+            const uint32_t actual_ip = na.as<const epee::net_utils::ipv4_network_address>().ip();
+            classB.insert(actual_ip & 0x0000ffff);
+          }
+          return true;
+        });
+      }
+
      std::deque<size_t> filtered;
      const size_t limit = use_white_list ? 20 : std::numeric_limits<size_t>::max();
-      size_t idx = 0;
-      zone.m_peerlist.foreach (use_white_list, [&filtered, &idx, limit, next_needed_pruning_stripe](const peerlist_entry &pe){
-        if (filtered.size() >= limit)
-          return false;
-        if (next_needed_pruning_stripe == 0 || pe.pruning_seed == 0)
-          filtered.push_back(idx);
-        else if (next_needed_pruning_stripe == tools::get_pruning_stripe(pe.pruning_seed))
-          filtered.push_front(idx);
-        ++idx;
-        return true;
-      });
+      size_t idx = 0, skipped = 0;
+      for (int step = 0; step < 2; ++step)
+      {
+        bool skip_duplicate_class_B = step == 0;
+        zone.m_peerlist.foreach (use_white_list, [&classB, &filtered, &idx, &skipped, skip_duplicate_class_B, limit, next_needed_pruning_stripe](const peerlist_entry &pe){
+          if (filtered.size() >= limit)
+            return false;
+          bool skip = false;
+          if (skip_duplicate_class_B && pe.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id())
+          {
+            const epee::net_utils::network_address na = pe.adr;
+            uint32_t actual_ip = na.as<const epee::net_utils::ipv4_network_address>().ip();
+            skip = classB.find(actual_ip & 0x0000ffff) != classB.end();
+          }
+          if (skip)
+            ++skipped;
+          else if (next_needed_pruning_stripe == 0 || pe.pruning_seed == 0)
+            filtered.push_back(idx);
+          else if (next_needed_pruning_stripe == tools::get_pruning_stripe(pe.pruning_seed))
+            filtered.push_front(idx);
+          ++idx;
+          return true;
+        });
+        if (skipped == 0 || !filtered.empty())
+          break;
+        if (skipped)
+          MGINFO("Skipping " << skipped << " possible peers as they share a class B with existing peers");
+      }
      if (filtered.empty())
      {
        MDEBUG("No available peer in " << (use_white_list ? "white" : "gray") << " list filtered by " << next_needed_pruning_stripe);
@@ -344,8 +344,14 @@ namespace nodetool
      trim_white_peerlist();
    }else
    {
-      //update record in white list 
-      m_peers_white.replace(by_addr_it_wt, ple);      
+      //update record in white list
+      peerlist_entry new_ple = ple;
+      if (by_addr_it_wt->pruning_seed && ple.pruning_seed == 0) // guard against older nodes not passing pruning info around
+        new_ple.pruning_seed = by_addr_it_wt->pruning_seed;
+      if (by_addr_it_wt->rpc_port && ple.rpc_port == 0) // guard against older nodes not passing RPC port around
+        new_ple.rpc_port = by_addr_it_wt->rpc_port;
+      new_ple.last_seen = by_addr_it_wt->last_seen; // do not overwrite the last seen timestamp, incoming peer list are untrusted
+      m_peers_white.replace(by_addr_it_wt, new_ple);
    }
    //remove from gray list, if need
    auto by_addr_it_gr = m_peers_gray.get<by_addr>().find(ple.adr);
@@ -379,8 +385,14 @@ namespace nodetool
      trim_gray_peerlist();    
    }else
    {
-      //update record in white list 
-      m_peers_gray.replace(by_addr_it_gr, ple);      
+      //update record in gray list
+      peerlist_entry new_ple = ple;
+      if (by_addr_it_gr->pruning_seed && ple.pruning_seed == 0) // guard against older nodes not passing pruning info around
+        new_ple.pruning_seed = by_addr_it_gr->pruning_seed;
+      if (by_addr_it_gr->rpc_port && ple.rpc_port == 0) // guard against older nodes not passing RPC port around
+        new_ple.rpc_port = by_addr_it_gr->rpc_port;
+      new_ple.last_seen = by_addr_it_gr->last_seen; // do not overwrite the last seen timestamp, incoming peer list are untrusted
+      m_peers_gray.replace(by_addr_it_gr, new_ple);
    }
    return true;
    CATCH_ENTRY_L0("peerlist_manager::append_with_peer_gray()", false);
@@ -134,10 +134,11 @@ namespace boost
      a & port;
      a & length;

-      if (length > net::tor_address::buffer_size())
+      const size_t buffer_size = net::tor_address::buffer_size();
+      if (length > buffer_size)
        MONERO_THROW(net::error::invalid_tor_address, "Tor address too long");

-      char host[net::tor_address::buffer_size()] = {0};
+      char host[buffer_size] = {0};
      a.load_binary(host, length);
      host[sizeof(host) - 1] = 0;

@@ -155,10 +156,11 @@ namespace boost
      a & port;
      a & length;

-      if (length > net::i2p_address::buffer_size())
+      const size_t buffer_size = net::i2p_address::buffer_size();
+      if (length > buffer_size)
        MONERO_THROW(net::error::invalid_i2p_address, "i2p address too long");

-      char host[net::i2p_address::buffer_size()] = {0};
+      char host[buffer_size] = {0};
      a.load_binary(host, length);
      host[sizeof(host) - 1] = 0;

@@ -252,7 +252,7 @@ namespace rct {
        {
          FIELD(type)
          if (type == RCTTypeNull)
-            return true;
+            return ar.stream().good();
          if (type != RCTTypeFull && type != RCTTypeSimple && type != RCTTypeBulletproof && type != RCTTypeBulletproof2)
            return false;
          VARINT_FIELD(txnFee)
@@ -312,7 +312,7 @@ namespace rct {
              ar.delimit_array();
          }
          ar.end_array();
-          return true;
+          return ar.stream().good();
        }
    };
    struct rctSigPrunable {
@@ -325,7 +325,7 @@ namespace rct {
        bool serialize_rctsig_prunable(Archive<W> &ar, uint8_t type, size_t inputs, size_t outputs, size_t mixin)
        {
          if (type == RCTTypeNull)
-            return true;
+            return ar.stream().good();
          if (type != RCTTypeFull && type != RCTTypeSimple && type != RCTTypeBulletproof && type != RCTTypeBulletproof2)
            return false;
          if (type == RCTTypeBulletproof || type == RCTTypeBulletproof2)
@@ -429,7 +429,7 @@ namespace rct {
            }
            ar.end_array();
          }
-          return true;
+          return ar.stream().good();
        }

    };
@@ -59,6 +59,8 @@ using namespace epee;
 #define MAX_RESTRICTED_FAKE_OUTS_COUNT 40
 #define MAX_RESTRICTED_GLOBAL_FAKE_OUTS_COUNT 5000

+#define OUTPUT_HISTOGRAM_RECENT_CUTOFF_RESTRICTION (3 * 86400) // 3 days max, the wallet requests 1.8 days
+
 namespace
 {
  void add_reason(std::string &reasons, const char *reason)
@@ -90,15 +92,9 @@ namespace cryptonote
    command_line::add_arg(desc, arg_rpc_bind_port);
    command_line::add_arg(desc, arg_rpc_restricted_bind_port);
    command_line::add_arg(desc, arg_restricted_rpc);
-    command_line::add_arg(desc, arg_rpc_ssl);
-    command_line::add_arg(desc, arg_rpc_ssl_private_key);
-    command_line::add_arg(desc, arg_rpc_ssl_certificate);
-    command_line::add_arg(desc, arg_rpc_ssl_ca_certificates);
-    command_line::add_arg(desc, arg_rpc_ssl_allowed_fingerprints);
-    command_line::add_arg(desc, arg_rpc_ssl_allow_any_cert);
    command_line::add_arg(desc, arg_bootstrap_daemon_address);
    command_line::add_arg(desc, arg_bootstrap_daemon_login);
-    cryptonote::rpc_args::init_options(desc);
+    cryptonote::rpc_args::init_options(desc, true);
  }
  //------------------------------------------------------------------------------------------------------------------------------
  core_rpc_server::core_rpc_server(
@@ -118,7 +114,7 @@ namespace cryptonote
    m_restricted = restricted;
    m_net_server.set_threads_prefix("RPC");

-    auto rpc_config = cryptonote::rpc_args::process(vm);
+    auto rpc_config = cryptonote::rpc_args::process(vm, true);
    if (!rpc_config)
      return false;

@@ -151,46 +147,9 @@ namespace cryptonote
    if (rpc_config->login)
      http_login.emplace(std::move(rpc_config->login->username), std::move(rpc_config->login->password).password());

-    epee::net_utils::ssl_options_t ssl_options = epee::net_utils::ssl_support_t::e_ssl_support_autodetect;
-    if (command_line::get_arg(vm, arg_rpc_ssl_allow_any_cert))
-      ssl_options.verification = epee::net_utils::ssl_verification_t::none;
-    else
-    {
-      std::string ssl_ca_path = command_line::get_arg(vm, arg_rpc_ssl_ca_certificates);
-      const std::vector<std::string> ssl_allowed_fingerprint_strings = command_line::get_arg(vm, arg_rpc_ssl_allowed_fingerprints);
-      std::vector<std::vector<uint8_t>> ssl_allowed_fingerprints{ ssl_allowed_fingerprint_strings.size() };
-      std::transform(ssl_allowed_fingerprint_strings.begin(), ssl_allowed_fingerprint_strings.end(), ssl_allowed_fingerprints.begin(), epee::from_hex::vector);
-      for (const auto &fpr: ssl_allowed_fingerprints)
-      {
-        if (fpr.size() != SSL_FINGERPRINT_SIZE)
-        {
-          MERROR("SHA-256 fingerprint should be " BOOST_PP_STRINGIZE(SSL_FINGERPRINT_SIZE) " bytes long.");
-          return false;
-        }
-      }
-
-      if (!ssl_ca_path.empty() || !ssl_allowed_fingerprints.empty())
-        ssl_options = epee::net_utils::ssl_options_t{std::move(ssl_allowed_fingerprints), std::move(ssl_ca_path)};
-    }
-
-    ssl_options.auth = epee::net_utils::ssl_authentication_t{
-      command_line::get_arg(vm, arg_rpc_ssl_private_key), command_line::get_arg(vm, arg_rpc_ssl_certificate)
-    };
-
-    // user specified CA file or fingeprints implies enabled SSL by default
-    if (ssl_options.verification != epee::net_utils::ssl_verification_t::user_certificates || !command_line::is_arg_defaulted(vm, arg_rpc_ssl))
-    {
-      const std::string ssl = command_line::get_arg(vm, arg_rpc_ssl);
-      if (!epee::net_utils::ssl_support_from_string(ssl_options.support, ssl))
-      {
-        MFATAL("Invalid RPC SSL support: " << ssl);
-        return false;
-      }
-    }
-
    auto rng = [](size_t len, uint8_t *ptr){ return crypto::rand(len, ptr); };
    return epee::http_server_impl_base<core_rpc_server, connection_context>::init(
-      rng, std::move(port), std::move(rpc_config->bind_ip), std::move(rpc_config->access_control_origins), std::move(http_login), std::move(ssl_options)
+      rng, std::move(port), std::move(rpc_config->bind_ip), std::move(rpc_config->access_control_origins), std::move(http_login), std::move(rpc_config->ssl_options)
    );
  }
  //------------------------------------------------------------------------------------------------------------------------------
@@ -861,6 +820,7 @@ namespace cryptonote
      res.sanity_check_failed = true;
      return true;
    }
+    res.sanity_check_failed = false;

    cryptonote_connection_context fake_context = AUTO_VAL_INIT(fake_context);
    tx_verification_context tvc = AUTO_VAL_INIT(tvc);
@@ -948,16 +908,13 @@ namespace cryptonote
      return true;
    }

-    boost::thread::attributes attrs;
-    attrs.set_stack_size(THREAD_STACK_SIZE);
-
    cryptonote::miner &miner= m_core.get_miner();
    if (miner.is_mining())
    {
      res.status = "Already mining";
      return true;
    }
-    if(!miner.start(info.address, static_cast<size_t>(req.threads_count), attrs, req.do_background_mining, req.ignore_battery))
+    if(!miner.start(info.address, static_cast<size_t>(req.threads_count), req.do_background_mining, req.ignore_battery))
    {
      res.status = "Failed, mining not started";
      LOG_PRINT_L0(res.status);
@@ -1928,6 +1885,13 @@ namespace cryptonote
    if (use_bootstrap_daemon_if_necessary<COMMAND_RPC_GET_OUTPUT_HISTOGRAM>(invoke_http_mode::JON_RPC, "get_output_histogram", req, res, r))
      return r;

+    const bool restricted = m_restricted && ctx;
+    if (restricted && req.recent_cutoff > 0 && req.recent_cutoff < (uint64_t)time(NULL) - OUTPUT_HISTOGRAM_RECENT_CUTOFF_RESTRICTION)
+    {
+      res.status = "Recent cutoff is too old";
+      return true;
+    }
+
    std::map<uint64_t, std::tuple<uint64_t, uint64_t, uint64_t>> histogram;
    try
    {
@@ -2439,40 +2403,6 @@ namespace cryptonote
    , false
    };

-  const command_line::arg_descriptor<std::string> core_rpc_server::arg_rpc_ssl = {
-      "rpc-ssl"
-    , "Enable SSL on RPC connections: enabled|disabled|autodetect"
-    , "autodetect"
-    };
-
-  const command_line::arg_descriptor<std::string> core_rpc_server::arg_rpc_ssl_private_key = {
-      "rpc-ssl-private-key"
-    , "Path to a PEM format private key"
-    , ""
-    };
-
-  const command_line::arg_descriptor<std::string> core_rpc_server::arg_rpc_ssl_certificate = {
-      "rpc-ssl-certificate"
-    , "Path to a PEM format certificate"
-    , ""
-    };
-
-  const command_line::arg_descriptor<std::string> core_rpc_server::arg_rpc_ssl_ca_certificates = {
-      "rpc-ssl-ca-certificates"
-    , "Path to file containing concatenated PEM format certificate(s) to replace system CA(s)."
-    };
-
-  const command_line::arg_descriptor<std::vector<std::string>> core_rpc_server::arg_rpc_ssl_allowed_fingerprints = {
-      "rpc-ssl-allowed-fingerprints"
-    , "List of certificate fingerprints to allow"
-  };
-
-  const command_line::arg_descriptor<bool> core_rpc_server::arg_rpc_ssl_allow_any_cert = {
-      "rpc-ssl-allow-any-cert"
-    , "Allow any peer certificate"
-    , false
-    };
-
  const command_line::arg_descriptor<std::string> core_rpc_server::arg_bootstrap_daemon_address = {
      "bootstrap-daemon-address"
    , "URL of a 'bootstrap' remote daemon that the connected wallets can use while this daemon is still not fully synced"
@@ -408,10 +408,7 @@ namespace rpc
      return;
    }

-    boost::thread::attributes attrs;
-    attrs.set_stack_size(THREAD_STACK_SIZE);
-
-    if(!m_core.get_miner().start(info.address, static_cast<size_t>(req.threads_count), attrs, req.do_background_mining, req.ignore_battery))
+    if(!m_core.get_miner().start(info.address, static_cast<size_t>(req.threads_count), req.do_background_mining, req.ignore_battery))
    {
      res.error_details = "Failed, mining not started";
      LOG_PRINT_L0(res.error_details);
@@ -33,28 +33,95 @@
 #include <boost/bind.hpp>
 #include "common/command_line.h"
 #include "common/i18n.h"
+#include "hex.h"

 namespace cryptonote
 {
+  namespace
+  {
+    boost::optional<epee::net_utils::ssl_options_t> do_process_ssl(const boost::program_options::variables_map& vm, const rpc_args::descriptors& arg, const bool any_cert_option)
+    {
+      bool ssl_required = false;
+      epee::net_utils::ssl_options_t ssl_options = epee::net_utils::ssl_support_t::e_ssl_support_enabled;
+      if (any_cert_option && command_line::get_arg(vm, arg.rpc_ssl_allow_any_cert))
+        ssl_options.verification = epee::net_utils::ssl_verification_t::none;
+      else
+      {
+        std::string ssl_ca_file = command_line::get_arg(vm, arg.rpc_ssl_ca_certificates);
+        const std::vector<std::string> ssl_allowed_fingerprints = command_line::get_arg(vm, arg.rpc_ssl_allowed_fingerprints);
+
+        std::vector<std::vector<uint8_t>> allowed_fingerprints{ ssl_allowed_fingerprints.size() };
+        std::transform(ssl_allowed_fingerprints.begin(), ssl_allowed_fingerprints.end(), allowed_fingerprints.begin(), epee::from_hex::vector);
+        for (const auto &fpr: allowed_fingerprints)
+        {
+          if (fpr.size() != SSL_FINGERPRINT_SIZE)
+          {
+            MERROR("SHA-256 fingerprint should be " BOOST_PP_STRINGIZE(SSL_FINGERPRINT_SIZE) " bytes long.");
+            return boost::none;
+          }
+        }
+
+        if (!allowed_fingerprints.empty() || !ssl_ca_file.empty())
+        {
+          ssl_required = true;
+          ssl_options = epee::net_utils::ssl_options_t{
+            std::move(allowed_fingerprints), std::move(ssl_ca_file)
+          };
+
+          if (command_line::get_arg(vm, arg.rpc_ssl_allow_chained))
+            ssl_options.verification = epee::net_utils::ssl_verification_t::user_ca;
+        }
+      }
+
+      // user specified CA file or fingeprints implies enabled SSL by default
+      if (!ssl_required && !epee::net_utils::ssl_support_from_string(ssl_options.support, command_line::get_arg(vm, arg.rpc_ssl)))
+      {
+        MERROR("Invalid argument for " << std::string(arg.rpc_ssl.name));
+        return boost::none;
+      }
+
+      ssl_options.auth = epee::net_utils::ssl_authentication_t{
+        command_line::get_arg(vm, arg.rpc_ssl_private_key), command_line::get_arg(vm, arg.rpc_ssl_certificate)
+      };
+
+      return {std::move(ssl_options)};
+    }
+  } // anonymous
+
  rpc_args::descriptors::descriptors()
     : rpc_bind_ip({"rpc-bind-ip", rpc_args::tr("Specify IP to bind RPC server"), "127.0.0.1"})
     , rpc_login({"rpc-login", rpc_args::tr("Specify username[:password] required for RPC server"), "", true})
     , confirm_external_bind({"confirm-external-bind", rpc_args::tr("Confirm rpc-bind-ip value is NOT a loopback (local) IP")})
     , rpc_access_control_origins({"rpc-access-control-origins", rpc_args::tr("Specify a comma separated list of origins to allow cross origin resource sharing"), ""})
+     , rpc_ssl({"rpc-ssl", rpc_args::tr("Enable SSL on RPC connections: enabled|disabled|autodetect"), "autodetect"})
+     , rpc_ssl_private_key({"rpc-ssl-private-key", rpc_args::tr("Path to a PEM format private key"), ""})
+     , rpc_ssl_certificate({"rpc-ssl-certificate", rpc_args::tr("Path to a PEM format certificate"), ""})
+     , rpc_ssl_ca_certificates({"rpc-ssl-ca-certificates", rpc_args::tr("Path to file containing concatenated PEM format certificate(s) to replace system CA(s)."), ""})
+     , rpc_ssl_allowed_fingerprints({"rpc-ssl-allowed-fingerprints", rpc_args::tr("List of certificate fingerprints to allow")})
+     , rpc_ssl_allow_chained({"rpc-ssl-allow-chained", rpc_args::tr("Allow user (via --rpc-ssl-certificates) chain certificates"), false})
+     , rpc_ssl_allow_any_cert({"rpc-ssl-allow-any-cert", rpc_args::tr("Allow any peer certificate"), false})
  {}

  const char* rpc_args::tr(const char* str) { return i18n_translate(str, "cryptonote::rpc_args"); }

-  void rpc_args::init_options(boost::program_options::options_description& desc)
+  void rpc_args::init_options(boost::program_options::options_description& desc, const bool any_cert_option)
  {
    const descriptors arg{};
    command_line::add_arg(desc, arg.rpc_bind_ip);
    command_line::add_arg(desc, arg.rpc_login);
    command_line::add_arg(desc, arg.confirm_external_bind);
    command_line::add_arg(desc, arg.rpc_access_control_origins);
+    command_line::add_arg(desc, arg.rpc_ssl);
+    command_line::add_arg(desc, arg.rpc_ssl_private_key);
+    command_line::add_arg(desc, arg.rpc_ssl_certificate);
+    command_line::add_arg(desc, arg.rpc_ssl_ca_certificates);
+    command_line::add_arg(desc, arg.rpc_ssl_allowed_fingerprints);
+    command_line::add_arg(desc, arg.rpc_ssl_allow_chained);
+    if (any_cert_option)
+      command_line::add_arg(desc, arg.rpc_ssl_allow_any_cert);
  }

-  boost::optional<rpc_args> rpc_args::process(const boost::program_options::variables_map& vm)
+  boost::optional<rpc_args> rpc_args::process(const boost::program_options::variables_map& vm, const bool any_cert_option)
  {
    const descriptors arg{};
    rpc_args config{};
@@ -118,6 +185,17 @@ namespace cryptonote
      config.access_control_origins = std::move(access_control_origins);
    }

+    auto ssl_options = do_process_ssl(vm, arg, any_cert_option);
+    if (!ssl_options)
+      return boost::none;
+    config.ssl_options = std::move(*ssl_options);
+
    return {std::move(config)};
  }
+
+  boost::optional<epee::net_utils::ssl_options_t> rpc_args::process_ssl(const boost::program_options::variables_map& vm, const bool any_cert_option)
+  {
+    const descriptors arg{};
+    return do_process_ssl(vm, arg, any_cert_option);
+  }
 }
@@ -35,6 +35,7 @@

 #include "common/command_line.h"
 #include "common/password.h"
+#include "net/net_ssl.h"

 namespace cryptonote
 {
@@ -54,16 +55,29 @@ namespace cryptonote
      const command_line::arg_descriptor<std::string> rpc_login;
      const command_line::arg_descriptor<bool> confirm_external_bind;
      const command_line::arg_descriptor<std::string> rpc_access_control_origins;
+      const command_line::arg_descriptor<std::string> rpc_ssl;
+      const command_line::arg_descriptor<std::string> rpc_ssl_private_key;
+      const command_line::arg_descriptor<std::string> rpc_ssl_certificate;
+      const command_line::arg_descriptor<std::string> rpc_ssl_ca_certificates;
+      const command_line::arg_descriptor<std::vector<std::string>> rpc_ssl_allowed_fingerprints;
+      const command_line::arg_descriptor<bool> rpc_ssl_allow_chained;
+      const command_line::arg_descriptor<bool> rpc_ssl_allow_any_cert;
    };

+    // `allow_any_cert` bool toggles `--rpc-ssl-allow-any-cert` configuration
+
    static const char* tr(const char* str);
-    static void init_options(boost::program_options::options_description& desc);
+    static void init_options(boost::program_options::options_description& desc, const bool any_cert_option = false);

    //! \return Arguments specified by user, or `boost::none` if error
-    static boost::optional<rpc_args> process(const boost::program_options::variables_map& vm);
+    static boost::optional<rpc_args> process(const boost::program_options::variables_map& vm, const bool any_cert_option = false);
+
+    //! \return SSL arguments specified by user, or `boost::none` if error
+    static boost::optional<epee::net_utils::ssl_options_t> process_ssl(const boost::program_options::variables_map& vm, const bool any_cert_option = false);

    std::string bind_ip;
    std::vector<std::string> access_control_origins;
    boost::optional<tools::login> login; // currently `boost::none` if unspecified by user
+    epee::net_utils::ssl_options_t ssl_options = epee::net_utils::ssl_support_t::e_ssl_support_enabled;
  };
 }
@@ -146,7 +146,8 @@ struct binary_archive<false> : public binary_archive_base<std::istream, false>
  void serialize_uvarint(T &v)
  {
    typedef std::istreambuf_iterator<char> it;
-    tools::read_varint(it(stream_), it(), v); // XXX handle failure
+    if (tools::read_varint(it(stream_), it(), v) < 0)
+      stream_.setstate(std::ios_base::failbit);
  }

  void begin_array(size_t &s)
@@ -212,7 +212,7 @@ inline bool do_serialize(Archive &ar, bool &v)
 * \brief self-explanatory
 */
 #define END_SERIALIZE()				\
-  return true;					\
+  return ar.stream().good();			\
  }

 /*! \macro VALUE(f)
@@ -1,5 +1,5 @@
 #define DEF_MONERO_VERSION_TAG "@VERSIONTAG@"
-#define DEF_MONERO_VERSION "0.14.1.0"
+#define DEF_MONERO_VERSION "0.14.1.2"
 #define DEF_MONERO_RELEASE_NAME "Boron Butterfly"
 #define DEF_MONERO_VERSION_FULL DEF_MONERO_VERSION "-" DEF_MONERO_VERSION_TAG

@@ -399,8 +399,11 @@ std::unique_ptr<tools::wallet2> make_basic(const boost::program_options::variabl
  {
    const boost::string_ref real_daemon = boost::string_ref{daemon_address}.substr(0, daemon_address.rfind(':'));

+    /* If SSL or proxy is enabled, then a specific cert, CA or fingerprint must
+       be specified. This is specific to the wallet. */
    const bool verification_required =
-      ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_enabled || use_proxy;
+      ssl_options.verification != epee::net_utils::ssl_verification_t::none &&
+      (ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_enabled || use_proxy);

    THROW_WALLET_EXCEPTION_IF(
      verification_required && !ssl_options.has_strong_verification(real_daemon),
@@ -66,11 +66,6 @@ namespace
  const command_line::arg_descriptor<bool> arg_restricted = {"restricted-rpc", "Restricts to view-only commands", false};
  const command_line::arg_descriptor<std::string> arg_wallet_dir = {"wallet-dir", "Directory for newly created wallets"};
  const command_line::arg_descriptor<bool> arg_prompt_for_password = {"prompt-for-password", "Prompts for password when not provided", false};
-  const command_line::arg_descriptor<std::string> arg_rpc_ssl = {"rpc-ssl", tools::wallet2::tr("Enable SSL on wallet RPC connections: enabled|disabled|autodetect"), "autodetect"};
-  const command_line::arg_descriptor<std::string> arg_rpc_ssl_private_key = {"rpc-ssl-private-key", tools::wallet2::tr("Path to a PEM format private key"), ""};
-  const command_line::arg_descriptor<std::string> arg_rpc_ssl_certificate = {"rpc-ssl-certificate", tools::wallet2::tr("Path to a PEM format certificate"), ""};
-  const command_line::arg_descriptor<std::string> arg_rpc_ssl_ca_certificates = {"rpc-ssl-ca-certificates", tools::wallet2::tr("Path to file containing concatenated PEM format certificate(s) to replace system CA(s).")};
-  const command_line::arg_descriptor<std::vector<std::string>> arg_rpc_ssl_allowed_fingerprints = {"rpc-ssl-allowed-fingerprints", tools::wallet2::tr("List of certificate fingerprints to allow")};

  constexpr const char default_rpc_username[] = "monero";

@@ -244,45 +239,6 @@ namespace tools
      assert(bool(http_login));
    } // end auth enabled

-    auto rpc_ssl_private_key = command_line::get_arg(vm, arg_rpc_ssl_private_key);
-    auto rpc_ssl_certificate = command_line::get_arg(vm, arg_rpc_ssl_certificate);
-    auto rpc_ssl_ca_file = command_line::get_arg(vm, arg_rpc_ssl_ca_certificates);
-    auto rpc_ssl_allowed_fingerprints = command_line::get_arg(vm, arg_rpc_ssl_allowed_fingerprints);
-    auto rpc_ssl = command_line::get_arg(vm, arg_rpc_ssl);
-    epee::net_utils::ssl_options_t rpc_ssl_options = epee::net_utils::ssl_support_t::e_ssl_support_enabled;
-
-    if (!rpc_ssl_ca_file.empty() || !rpc_ssl_allowed_fingerprints.empty())
-    {
-      std::vector<std::vector<uint8_t>> allowed_fingerprints{ rpc_ssl_allowed_fingerprints.size() };
-      std::transform(rpc_ssl_allowed_fingerprints.begin(), rpc_ssl_allowed_fingerprints.end(), allowed_fingerprints.begin(), epee::from_hex::vector);
-      for (const auto &fpr: allowed_fingerprints)
-      {
-        if (fpr.size() != SSL_FINGERPRINT_SIZE)
-        {
-          MERROR("SHA-256 fingerprint should be " BOOST_PP_STRINGIZE(SSL_FINGERPRINT_SIZE) " bytes long.");
-          return false;
-        }
-      }
-
-      rpc_ssl_options = epee::net_utils::ssl_options_t{
-        std::move(allowed_fingerprints), std::move(rpc_ssl_ca_file)
-      };
-    }
-
-    // user specified CA file or fingeprints implies enabled SSL by default
-    if (rpc_ssl_options.verification != epee::net_utils::ssl_verification_t::user_certificates || !command_line::is_arg_defaulted(vm, arg_rpc_ssl))
-    {
-      if (!epee::net_utils::ssl_support_from_string(rpc_ssl_options.support, rpc_ssl))
-      {
-        MERROR("Invalid argument for " << std::string(arg_rpc_ssl.name));
-        return false;
-      }
-    }
-
-    rpc_ssl_options.auth = epee::net_utils::ssl_authentication_t{
-      std::move(rpc_ssl_private_key), std::move(rpc_ssl_certificate)
-    };
-
    m_auto_refresh_period = DEFAULT_AUTO_REFRESH_PERIOD;
    m_last_auto_refresh_time = boost::posix_time::min_date_time;

@@ -292,7 +248,7 @@ namespace tools
    auto rng = [](size_t len, uint8_t *ptr) { return crypto::rand(len, ptr); };
    return epee::http_server_impl_base<wallet_rpc_server, connection_context>::init(
      rng, std::move(bind_port), std::move(rpc_config->bind_ip), std::move(rpc_config->access_control_origins), std::move(http_login),
-      std::move(rpc_ssl_options)
+      std::move(rpc_config->ssl_options)
    );
  }
  //------------------------------------------------------------------------------------------------------------------------------
@@ -1789,6 +1745,11 @@ namespace tools
      else if (payment_id_str.size() == 2 * sizeof(payment_id8))
      {
        r = epee::string_tools::hex_to_pod(payment_id_str, payment_id8);
+        if (r)
+        {
+          memcpy(payment_id.data, payment_id8.data, 8);
+          memset(payment_id.data + 8, 0, 24);
+        }
      }
      else
      {
@@ -4167,7 +4128,11 @@ namespace tools
      std::move(req.ssl_private_key_path), std::move(req.ssl_certificate_path)
    };

-    if (ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_enabled && !ssl_options.has_strong_verification(boost::string_ref{}))
+    const bool verification_required =
+      ssl_options.verification != epee::net_utils::ssl_verification_t::none &&
+      ssl_options.support == epee::net_utils::ssl_support_t::e_ssl_support_enabled;
+
+    if (verification_required && !ssl_options.has_strong_verification(boost::string_ref{}))
    {
      er.code = WALLET_RPC_ERROR_CODE_NO_DAEMON_CONNECTION;
      er.message = "SSL is enabled but no user certificate or fingerprints were provided";
@@ -4412,11 +4377,6 @@ int main(int argc, char** argv) {
  command_line::add_arg(desc_params, arg_from_json);
  command_line::add_arg(desc_params, arg_wallet_dir);
  command_line::add_arg(desc_params, arg_prompt_for_password);
-  command_line::add_arg(desc_params, arg_rpc_ssl);
-  command_line::add_arg(desc_params, arg_rpc_ssl_private_key);
-  command_line::add_arg(desc_params, arg_rpc_ssl_certificate);
-  command_line::add_arg(desc_params, arg_rpc_ssl_ca_certificates);
-  command_line::add_arg(desc_params, arg_rpc_ssl_allowed_fingerprints);

  daemonizer::init_options(hidden_options, desc_params);
  desc_params.add(hidden_options);
@@ -28,6 +28,7 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

+from __future__ import print_function
 import time

 """Test peer baning RPC calls
@@ -42,7 +43,7 @@ from framework.daemon import Daemon

 class BanTest():
    def run_test(self):
-        print 'Testing bans'
+        print('Testing bans')

        daemon = Daemon()
        res = daemon.get_bans()
@@ -28,6 +28,7 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

+from __future__ import print_function
 import time

 """Test daemon blockchain RPC calls
@@ -50,7 +51,7 @@ class BlockchainTest():
        self._test_alt_chains()

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()
@@ -58,7 +59,7 @@ class BlockchainTest():
    def _test_generateblocks(self, blocks):
        assert blocks >= 2

-        print "Test generating", blocks, 'blocks'
+        print("Test generating", blocks, 'blocks')

        daemon = Daemon()

@@ -182,14 +183,14 @@ class BlockchainTest():
                for idx in tx.output_indices:
                    assert idx == running_output_index
                    running_output_index += 1
-                res_out = daemon.get_outs([{'amount': 0, 'index': i} for i in tx.output_indices], get_txid = True)
+                res_out = daemon.get_outs([{'amount': 0, 'index': idx} for idx in tx.output_indices], get_txid = True)
                assert len(res_out.outs) == len(tx.output_indices)
                for out in res_out.outs:
                    assert len(out.key) == 64
                    assert len(out.mask) == 64
                    assert not out.unlocked
-                    assert out.height == i + 1
-                    assert out.txid == txids[i + 1]
+                    assert out.height == i
+                    assert out.txid == txids[i]

        for i in range(height + nblocks - 1):
            res_sum = daemon.get_coinbase_tx_sum(i, 1)
@@ -255,7 +256,7 @@ class BlockchainTest():
            alt_blocks[i] = txid
            nonce += 1

-        print 'mining 3 on 1'
+        print('mining 3 on 1')
        # three more on [1]
        chain1 = []
        res = daemon.generateblocks('42ey1afDFnn4886T7196doS9GPMzexD9gXpsZJDwVjeRVdFCSoHnv7KPbBeGpzJBzHRCAs9UxqeoyFQMYbqSWYTfJJQAWDm', 3, prev_block = alt_blocks[1], starting_nonce = nonce)
@@ -275,7 +276,7 @@ class BlockchainTest():
        for txid in alt_blocks:
            assert txid in res.blks_hashes or txid == alt_blocks[1]

-        print 'mining 4 on 3'
+        print('mining 4 on 3')
        # 4 more on [3], the chain will reorg when we mine the 4th
        top_block_hash = blk_hash
        prev_block = alt_blocks[3]
@@ -28,11 +28,10 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
-
 """Test cold tx signing
 """

+from __future__ import print_function
 from framework.daemon import Daemon
 from framework.wallet import Wallet

@@ -44,13 +43,13 @@ class ColdSigningTest():
        self.transfer()

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()

    def create(self, idx):
-        print 'Creating hot and cold wallet'
+        print('Creating hot and cold wallet')

        self.hot_wallet = Wallet(idx = 0)
        # close the wallet if any, will throw if none is loaded
@@ -116,7 +115,7 @@ class ColdSigningTest():
        assert len(res.unsigned_txset) > 0
        unsigned_txset = res.unsigned_txset

-        print 'Signing transaction with cold wallet'
+        print('Signing transaction with cold wallet')
        res = self.cold_wallet.describe_transfer(unsigned_txset = unsigned_txset)
        assert len(res.desc) == 1
        desc = res.desc[0]
@@ -140,7 +139,7 @@ class ColdSigningTest():
        txid = res.tx_hash_list[0]
        assert len(txid) == 64

-        print 'Submitting transaction with hot wallet'
+        print('Submitting transaction with hot wallet')
        res = self.hot_wallet.submit_transfer(signed_txset)
        assert len(res.tx_hash_list) > 0
        assert res.tx_hash_list[0] == txid
@@ -36,6 +36,7 @@ Test the following RPCs:

 """

+from __future__ import print_function
 from framework.daemon import Daemon

 class DaemonGetInfoTest():
@@ -10,7 +10,7 @@ import string
 import os

 USAGE = 'usage: functional_tests_rpc.py <python> <srcdir> <builddir> [<tests-to-run> | all]'
-DEFAULT_TESTS = ['daemon_info', 'blockchain', 'wallet_address', 'integrated_address', 'mining', 'transfer', 'txpool', 'multisig', 'cold_signing', 'sign_message', 'proofs', 'get_output_distribution']
+DEFAULT_TESTS = ['bans', 'daemon_info', 'blockchain', 'wallet_address', 'integrated_address', 'mining', 'transfer', 'txpool', 'multisig', 'cold_signing', 'sign_message', 'proofs', 'get_output_distribution']
 try:
  python = sys.argv[1]
  srcdir = sys.argv[2]
@@ -28,11 +28,10 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
-
 """Test get_output_distribution RPC
 """

+from __future__ import print_function
 from framework.daemon import Daemon
 from framework.wallet import Wallet

@@ -43,7 +42,7 @@ class GetOutputDistributionTest():
        self.test_get_output_distribution()

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()
@@ -56,7 +55,7 @@ class GetOutputDistributionTest():
        res = self.wallet.restore_deterministic_wallet(seed = 'velvet lymph giddy number token physics poetry unquoted nibs useful sabotage limits benches lifestyle eden nitrogen anvil fewest avoid batch vials washing fences goat unquoted')

    def test_get_output_distribution(self):
-        print "Test get_output_distribution"
+        print("Test get_output_distribution")

        daemon = Daemon()

@@ -213,5 +212,14 @@ class GetOutputDistributionTest():
            assert d.distribution[h] == 0


+class Guard:
+    def __enter__(self):
+        for i in range(4):
+            Wallet(idx = i).auto_refresh(False)
+    def __exit__(self, exc_type, exc_value, traceback):
+        for i in range(4):
+            Wallet(idx = i).auto_refresh(True)
+
 if __name__ == '__main__':
-    GetOutputDistributionTest().run_test()
+    with Guard() as guard:
+        GetOutputDistributionTest().run_test()
@@ -28,8 +28,6 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
-
 """Test integrated address RPC calls

 Test the following RPCs:
@@ -38,6 +36,7 @@ Test the following RPCs:

 """

+from __future__ import print_function
 from framework.wallet import Wallet

 class IntegratedAddressTest():
@@ -46,7 +45,7 @@ class IntegratedAddressTest():
      self.check()

    def create(self):
-        print 'Creating wallet'
+        print('Creating wallet')
        wallet = Wallet()
        # close the wallet if any, will throw if none is loaded
        try: wallet.close_wallet()
@@ -59,7 +58,7 @@ class IntegratedAddressTest():
    def check(self):
        wallet = Wallet()

-        print 'Checking local address'
+        print('Checking local address')
        res = wallet.make_integrated_address(payment_id = '0123456789abcdef')
        assert res.integrated_address == '4CMe2PUhs4J4886T7196doS9GPMzexD9gXpsZJDwVjeRVdFCSoHnv7KPbBeGpzJBzHRCAs9UxqeoyFQMYbqSWYTfSbLRB61BQVATzerHGj'
        assert res.payment_id == '0123456789abcdef'
@@ -67,7 +66,7 @@ class IntegratedAddressTest():
        assert res.standard_address == '42ey1afDFnn4886T7196doS9GPMzexD9gXpsZJDwVjeRVdFCSoHnv7KPbBeGpzJBzHRCAs9UxqeoyFQMYbqSWYTfJJQAWDm'
        assert res.payment_id == '0123456789abcdef'

-        print 'Checking different address'
+        print('Checking different address')
        res = wallet.make_integrated_address(standard_address = '46r4nYSevkfBUMhuykdK3gQ98XDqDTYW1hNLaXNvjpsJaSbNtdXh1sKMsdVgqkaihChAzEy29zEDPMR3NHQvGoZCLGwTerK', payment_id = '1122334455667788')
        assert res.integrated_address == '4GYjoMG9Y2BBUMhuykdK3gQ98XDqDTYW1hNLaXNvjpsJaSbNtdXh1sKMsdVgqkaihChAzEy29zEDPMR3NHQvGoZCVSs1ZojwrDCGS5rUuo'
        assert res.payment_id == '1122334455667788'
@@ -75,7 +74,7 @@ class IntegratedAddressTest():
        assert res.standard_address == '46r4nYSevkfBUMhuykdK3gQ98XDqDTYW1hNLaXNvjpsJaSbNtdXh1sKMsdVgqkaihChAzEy29zEDPMR3NHQvGoZCLGwTerK'
        assert res.payment_id == '1122334455667788'

-        print 'Checking bad payment id'
+        print('Checking bad payment id')
        fails = 0
        try: wallet.make_integrated_address(standard_address = '46r4nYSevkfBUMhuykdK3gQ98XDqDTYW1hNLaXNvjpsJaSbNtdXh1sKMsdVgqkaihChAzEy29zEDPMR3NHQvGoZCLGwTerK', payment_id = '11223344556677880')
        except: fails += 1
@@ -89,7 +88,7 @@ class IntegratedAddressTest():
        except: fails += 1
        assert fails == 5

-        print 'Checking bad standard address'
+        print('Checking bad standard address')
        fails = 0
        try: wallet.make_integrated_address(standard_address = '46r4nYSevkfBUMhuykdK3gQ98XDqDTYW1hNLaXNvjpsJaSbNtdXh1sKMsdVgqkaihChAzEy29zEDPMR3NHQvGoZCLGwTerr', payment_id = '1122334455667788')
        except: fails += 1
@@ -28,6 +28,7 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

+from __future__ import print_function
 import time

 """Test daemon mining RPC calls
@@ -48,13 +49,13 @@ class MiningTest():
        self.mine()

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()

    def create(self):
-        print 'Creating wallet'
+        print('Creating wallet')
        wallet = Wallet()
        # close the wallet if any, will throw if none is loaded
        try: wallet.close_wallet()
@@ -62,7 +63,7 @@ class MiningTest():
        res = wallet.restore_deterministic_wallet(seed = 'velvet lymph giddy number token physics poetry unquoted nibs useful sabotage limits benches lifestyle eden nitrogen anvil fewest avoid batch vials washing fences goat unquoted')

    def mine(self):
-        print "Test mining"
+        print("Test mining")

        daemon = Daemon()
        wallet = Wallet()
@@ -28,7 +28,7 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
+from __future__ import print_function

 """Test multisig transfers
 """
@@ -70,7 +70,7 @@ class MultisigTest():
        self.check_transaction(txid)

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()
@@ -28,7 +28,7 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
+from __future__ import print_function

 """Test misc proofs (tx key, send, receive, reserve)
 """
@@ -47,7 +47,7 @@ class ProofsTest():
        self.check_reserve_proof()

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()
@@ -28,7 +28,7 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
+from __future__ import print_function

 """Test message signing/verification RPC calls

@@ -46,7 +46,7 @@ class MessageSigningTest():
      self.check_signing()

    def create(self):
-        print 'Creating wallets'
+        print('Creating wallets')
        seeds = [
            'velvet lymph giddy number token physics poetry unquoted nibs useful sabotage limits benches lifestyle eden nitrogen anvil fewest avoid batch vials washing fences goat unquoted',
            'peeled mixture ionic radar utopia puddle buying illness nuns gadget river spout cavernous bounced paradise drunk looking cottage jump tequila melting went winter adjust spout',
@@ -66,7 +66,7 @@ class MessageSigningTest():
            assert res.seed == seeds[i]

    def check_signing(self):
-        print 'Signing/verifing messages'
+        print('Signing/verifing messages')
        messages = ['foo', '']
        for message in messages:
            res = self.wallet[0].sign(message)
@@ -40,7 +40,7 @@ Test the following RPCs:

 import time
 from time import sleep
-from decimal import Decimal
+from __future__ import print_function

 from framework.daemon import Daemon
 from framework.wallet import Wallet
@@ -28,7 +28,7 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
+from __future__ import print_function
 import json

 """Test simple transfers
@@ -48,13 +48,13 @@ class TransferTest():
        self.sweep_single()

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()

    def create(self):
-        print 'Creating wallets'
+        print('Creating wallets')
        seeds = [
          'velvet lymph giddy number token physics poetry unquoted nibs useful sabotage limits benches lifestyle eden nitrogen anvil fewest avoid batch vials washing fences goat unquoted',
          'peeled mixture ionic radar utopia puddle buying illness nuns gadget river spout cavernous bounced paradise drunk looking cottage jump tequila melting went winter adjust spout',
@@ -297,7 +297,7 @@ class TransferTest():
        assert res.unlocked_balance <= res.balance
        assert res.blocks_to_unlock == 9

-        print 'Creating multi out transfer'
+        print('Creating multi out transfer')

        self.wallet[0].refresh()

@@ -519,6 +519,9 @@ class TransferTest():
        res = self.wallet[2].get_bulk_payments(payment_ids = ['1'*64, '1234500000012345abcde00000abcdeff1234500000012345abcde00000abcde', '2'*64])
        assert len(res.payments) >= 1 # one tx was sent

+        res = self.wallet[1].get_bulk_payments(["1111111122222222"])
+        assert len(res.payments) >= 1 # we have one of these
+
    def check_double_spend_detection(self):
        print('Checking double spend detection')
        txes = [[None, None], [None, None]]
@@ -28,7 +28,7 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
+from __future__ import print_function

 """Test txpool
 """
@@ -44,13 +44,13 @@ class TransferTest():
        self.check_txpool()

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()

    def create(self):
-        print 'Creating wallet'
+        print('Creating wallet')
        wallet = Wallet()
        # close the wallet if any, will throw if none is loaded
        try: wallet.close_wallet()
@@ -114,15 +114,16 @@ class TransferTest():
        assert sorted(res.tx_hashes) == sorted(txes.keys())

        print('Flushing 2 transactions')
-        daemon.flush_txpool([txes.keys()[1], txes.keys()[3]])
+        txes_keys = list(txes.keys())
+        daemon.flush_txpool([txes_keys[1], txes_keys[3]])
        res = daemon.get_transaction_pool()
        assert len(res.transactions) == txpool_size - 2
-        assert len([x for x in res.transactions if x.id_hash == txes.keys()[1]]) == 0
-        assert len([x for x in res.transactions if x.id_hash == txes.keys()[3]]) == 0
+        assert len([x for x in res.transactions if x.id_hash == txes_keys[1]]) == 0
+        assert len([x for x in res.transactions if x.id_hash == txes_keys[3]]) == 0

-        new_keys = txes.keys()
-        new_keys.remove(txes.keys()[1])
-        new_keys.remove(txes.keys()[3])
+        new_keys = list(txes.keys())
+        new_keys.remove(txes_keys[1])
+        new_keys.remove(txes_keys[3])
        res = daemon.get_transaction_pool_hashes()
        assert sorted(res.tx_hashes) == sorted(new_keys)

@@ -29,8 +29,6 @@
 # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
 # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-import time
-
 """Test transaction creation RPC calls

 Test the following RPCs:
@@ -38,6 +36,7 @@ Test the following RPCs:

 """

+from __future__ import print_function
 from framework.wallet import Wallet
 from framework.daemon import Daemon

@@ -52,13 +51,13 @@ class WalletAddressTest():
      self.languages()

    def reset(self):
-        print 'Resetting blockchain'
+        print('Resetting blockchain')
        daemon = Daemon()
        daemon.pop_blocks(1000)
        daemon.flush_txpool()

    def create(self):
-        print 'Creating wallet'
+        print('Creating wallet')
        wallet = Wallet()
        # close the wallet if any, will throw if none is loaded
        try: wallet.close_wallet()
@@ -69,7 +68,7 @@ class WalletAddressTest():
        assert res.seed == seed

    def check_main_address(self):
-        print 'Getting address'
+        print('Getting address')
        wallet = Wallet()
        res = wallet.get_address()
        assert res.address == '42ey1afDFnn4886T7196doS9GPMzexD9gXpsZJDwVjeRVdFCSoHnv7KPbBeGpzJBzHRCAs9UxqeoyFQMYbqSWYTfJJQAWDm', res
@@ -79,7 +78,7 @@ class WalletAddressTest():
        assert res.addresses[0].used == False

    def check_keys(self):
-        print 'Checking keys'
+        print('Checking keys')
        wallet = Wallet()
        res = wallet.query_key('view_key')
        assert res.key == '49774391fa5e8d249fc2c5b45dadef13534bf2483dede880dac88f061e809100'
@@ -89,7 +88,7 @@ class WalletAddressTest():
        assert res.key == 'velvet lymph giddy number token physics poetry unquoted nibs useful sabotage limits benches lifestyle eden nitrogen anvil fewest avoid batch vials washing fences goat unquoted'

    def create_subaddresses(self):
-        print 'Creating subaddresses'
+        print('Creating subaddresses')
        wallet = Wallet()
        res = wallet.create_account("idx1")
        assert res.account_index == 1, res
@@ -160,7 +159,7 @@ class WalletAddressTest():
        assert res.index == {'major': 1, 'minor': 0}

    def open_close(self):
-        print 'Testing open/close'
+        print('Testing open/close')
        wallet = Wallet()

        res = wallet.get_address()
@@ -200,7 +199,7 @@ class WalletAddressTest():
        except: pass
        languages = res.languages
        for language in languages:
-            print 'Creating ' + str(language) + ' wallet'
+            print('Creating ' + str(language) + ' wallet')
            wallet.create_wallet(filename = '', language = language)
            res = wallet.query_key('mnemonic')
            wallet.close_wallet()
@@ -148,3 +148,20 @@ TEST(keccak, 137_and_1_136)
  TEST_KECCAK(137, chunks);
 }

+TEST(keccak, alignment)
+{
+  uint8_t data[6064];
+  __attribute__ ((aligned(16))) char adata[6000];
+
+  for (size_t i = 0; i < sizeof(data) / sizeof(data[0]); ++i)
+    data[i] = i & 1;
+
+  uint8_t md[32], amd[32];
+  for (int offset = 0; offset < 64; ++offset)
+  {
+    memcpy(adata, data + offset, 6000);
+    keccak((const uint8_t*)&data[offset], 6000, md, 32);
+    keccak((const uint8_t*)adata, 6000, amd, 32);
+    ASSERT_TRUE(!memcmp(md, amd, 32));
+  }
+}
@@ -172,7 +172,7 @@ TEST(select_outputs, density)
    float chain_ratio = count_chain / (float)n_outs;
    MDEBUG(count_selected << "/" << NPICKS << " outputs selected in blocks of density " << d << ", " << 100.0f * selected_ratio << "%");
    MDEBUG(count_chain << "/" << offsets.size() << " outputs in blocks of density " << d << ", " << 100.0f * chain_ratio << "%");
-    ASSERT_LT(fabsf(selected_ratio - chain_ratio), 0.02f);
+    ASSERT_LT(fabsf(selected_ratio - chain_ratio), 0.025f);
  }
 }
Author	SHA1	Message	Date
Riccardo Spagni	8f0aedfa1a	Merge pull request #5762 `0a9bda81` bump version to 0.14.1.2 (Riccardo Spagni) `f4f00acc` update checkpoints.dat hash (Riccardo Spagni)	2019-07-18 17:18:13 +02:00
Riccardo Spagni	0a9bda813d	bump version to 0.14.1.2	2019-07-18 16:56:57 +02:00
Riccardo Spagni	f4f00accdb	update checkpoints.dat hash	2019-07-18 16:54:25 +02:00
Riccardo Spagni	da8ffdfff0	Merge pull request #5760 `65e22229` prep for 0.14.1.1 (Riccardo Spagni)	2019-07-17 22:40:18 +02:00
Riccardo Spagni	65e2222902	prep for 0.14.1.1	2019-07-17 22:25:13 +02:00
Riccardo Spagni	3aa87ad178	Merge pull request #5733 `1542d5d9` cryptonote_protocol: drop peers we can't download from when syncing (moneromooo-monero) `ff42af87` p2p: add a few missing connection close calls (moneromooo-monero)	2019-07-17 21:19:24 +02:00
luigi1111	5e81214d5c	Merge pull request #5747 `f3febaf` wallet_rpc_server: fix get_bulk_payments with short payment ids (moneromooo-monero)	2019-07-12 20:29:32 -05:00
luigi1111	2532f5a9e6	Merge pull request #5737 `8072e23` p2p: don't connect to more than one IP per class B if we can (moneromooo-monero)	2019-07-12 20:25:51 -05:00
luigi1111	d89b6226ad	Merge pull request #5726 `200e53c` keccak: guard against misaligned memory accesses on ARM (moneromooo-monero)	2019-07-12 20:24:05 -05:00
luigi1111	b08cacb677	Merge pull request #5703 `6461815` Depends: Update HIDAPI version (TheCharlatan)	2019-07-12 20:20:53 -05:00
luigi1111	fffd616cbe	Merge pull request #5694 `5157060` Add debug targets to depends Makefile (TheCharlatan) `8475cbb` Remove clutter in depends installed packages (TheCharlatan) `48c2659` Add ncurses package for linux and darwin readline (TheCharlatan)	2019-07-12 20:19:57 -05:00
moneromooo-monero	f3febafe80	wallet_rpc_server: fix get_bulk_payments with short payment ids	2019-07-09 14:39:10 +00:00
moneromooo-monero	8072e23249	p2p: don't connect to more than one IP per class B if we can	2019-07-05 18:25:34 +00:00
moneromooo-monero	200e53cfdd	keccak: guard against misaligned memory accesses on ARM The code generated is exactly the same as the direct access one on x86_64	2019-07-04 22:13:57 +00:00
moneromooo-monero	1542d5d978	cryptonote_protocol: drop peers we can't download from when syncing Any peer that's behind us while syncing is useless to us (though not to them). This ensures that we don't get our peer slots filled with peers that we can't use. Once we've synced, we can connect to them and they can then sync off us if they want.	2019-07-04 21:57:05 +00:00
moneromooo-monero	ff42af873c	p2p: add a few missing connection close calls	2019-07-04 21:57:02 +00:00
luigi1111	bb3c9ec403	Merge pull request #5664 `c1eba2b` core: fix --prune-blockchain not pruning if no blockchain exists (moneromooo-monero)	2019-07-04 14:47:59 -05:00
luigi1111	0825409608	Merge pull request #5651 `c382677` blockchain: silence an error getting blocks for pruned nodes (moneromooo-monero)	2019-07-04 14:45:28 -05:00
luigi1111	ef4429e9ec	Merge pull request #5666 `ba035bf` db_lmdb: commit pruning txn at checkpoints (moneromooo-monero)	2019-07-04 14:43:15 -05:00
luigi1111	2af1ea5cb5	Merge pull request #5708 `c78f7c5` Remove Xiala.net from the list of dns resolvers (tobtoht)	2019-07-01 02:08:17 -05:00
luigi1111	6e3d6f601a	Merge pull request #5684 `93eb4d7` rpc: set sanity_check_failed to false when successful (stoffu)	2019-07-01 00:23:18 -05:00
luigi1111	124ea419f0	Merge pull request #5676 `bb7b059` tx_sanity_check: relax the median check a lot (monermooo-monero)	2019-07-01 00:20:21 -05:00
luigi1111	ae77f09e68	Merge pull request #5669 `761a39a` p2p: don't forget pruning seed or public RPC port when updating peers (moneromooo-monero)	2019-07-01 00:15:39 -05:00
tobtoht	c78f7c58e1	Remove Xiala.net from the list of dns resolvers It is down permanently. See: https://xiala.net/ "Ende November 2018 werden alle Dienste von xiala.net abgeschaltet."	2019-06-28 20:45:36 +00:00
TheCharlatan	64618150ec	Depends: Update HIDAPI version The macos binaries in release v0.14.1.0 were compiled with the buggy hidapi-0.8.0-rc1 version. This resulted in users not being able to use their Ledger with the latest cli wallet. After the patch depends now fetches the source from the libusb hidapi repository that has taken over maintenance of hidapi.	2019-06-27 17:30:15 +02:00
moneromooo-monero	761a39a9de	p2p: don't forget pruning seed or public RPC port when updating peers Older nodes don't pass that information around	2019-06-26 10:34:51 +00:00
TheCharlatan	48c26599e6	Add ncurses package for linux and darwin readline Readline support is now compiled with the ncurses backend.	2019-06-25 19:16:59 +02:00
TheCharlatan	8475cbbd90	Remove clutter in depends installed packages To speedup the depends cached builds, remove some some clutter from the package files. This mainly incldues removing all the shared libraries and .la linker files. It also gives stronger guarantees that monero only links the static libs without any external rvalues.	2019-06-25 19:16:59 +02:00
TheCharlatan	5157060077	Add debug targets to depends Makefile Packages can now be built individually and for each stage. This allows easier debugging.	2019-06-25 19:16:59 +02:00
stoffu	93eb4d79b4	rpc: set sanity_check_failed to false when successful	2019-06-22 07:43:36 +09:00
moneromooo-monero	bb7b059d70	tx_sanity_check: relax the median check a lot we don't want to prevent bona fide txes, just obvious bad ones	2019-06-19 23:31:54 +00:00
moneromooo-monero	ba035bf827	db_lmdb: commit pruning txn at checkpoints to avoid errors when the txn is too large	2019-06-18 10:31:38 +00:00
moneromooo-monero	c1eba2b587	core: fix --prune-blockchain not pruning if no blockchain exists	2019-06-17 16:27:15 +00:00
moneromooo-monero	c382677ee3	blockchain: silence an error getting blocks for pruned nodes This happens often when a pre-pruning node asks a pruned node for data it does not have	2019-06-16 11:03:14 +00:00
Riccardo Spagni	29a505d1c1	Merge pull request #5638 `6ed1679b` prep for 0.14.1 release (Riccardo Spagni)	2019-06-14 16:09:16 +02:00
Riccardo Spagni	c58255ec12	Merge pull request #5640 `542cab02` rpc: restrict the recent cutoff size in restricted RPC mode (moneromooo-monero) `434e617a` ensure no NULL is passed to memcpy (moneromooo-monero) `279f1f2c` abstract_tcp_server2: improve DoS resistance (moneromooo-monero) `756773e5` serialization: check stream good flag at the end (moneromooo-monero) `e3f714aa` tree-hash: allocate variable memory on heap, not stack (moneromooo-monero) `67baa3a6` cryptonote: throw on tx hash calculation error (moneromooo-monero) `d6bb9ecc` serialization: fail on read_varint error (moneromooo-monero) `19490e44` cryptonote_protocol: fix another potential P2P DoS (moneromooo-monero) `fa4aa47e` cryptonote_protocol: expand basic DoS protection (moneromooo-monero) `3c953d53` cryptonote_protocol_handler: prevent potential DoS (anonimal) `b873b69d` epee: basic sanity check on allocation size from untrusted source (moneromooo-monero)	2019-06-14 16:08:59 +02:00
moneromooo-monero	542cab02e1	rpc: restrict the recent cutoff size in restricted RPC mode	2019-06-14 08:48:27 +00:00
moneromooo-monero	434e617a1d	ensure no NULL is passed to memcpy NULL is valid when size is 0, but memcpy uses nonnull attributes, so let's not poke the bear	2019-06-14 08:48:25 +00:00
moneromooo-monero	279f1f2c26	abstract_tcp_server2: improve DoS resistance	2019-06-14 08:48:22 +00:00
moneromooo-monero	756773e5fe	serialization: check stream good flag at the end just in case	2019-06-14 08:48:19 +00:00
moneromooo-monero	e3f714aa2a	tree-hash: allocate variable memory on heap, not stack Large amounts might run out of stack Reported by guidov	2019-06-14 08:48:16 +00:00
moneromooo-monero	67baa3a66b	cryptonote: throw on tx hash calculation error	2019-06-14 08:48:13 +00:00
moneromooo-monero	d6bb9ecc6f	serialization: fail on read_varint error	2019-06-14 08:48:10 +00:00
moneromooo-monero	19490e44af	cryptonote_protocol: fix another potential P2P DoS When asking for txes in a fluffy transaction, one might ask for the same (large) tx many times	2019-06-14 08:48:07 +00:00
moneromooo-monero	fa4aa47ea0	cryptonote_protocol: expand basic DoS protection Count transactions as well	2019-06-14 08:48:04 +00:00
anonimal	3c953d5369	cryptonote_protocol_handler: prevent potential DoS Essentially, one can send such a large amount of IDs that core exhausts all free memory. This issue can theoretically be exploited using very large CN blockchains, such as Monero. This is a partial fix. Thanks and credit given to CryptoNote author 'cryptozoidberg' for collaboration and the fix. Also thanks to 'moneromooo'. Referencing HackerOne report #506595.	2019-06-14 08:48:01 +00:00
moneromooo-monero	b873b69ded	epee: basic sanity check on allocation size from untrusted source Reported by guidov	2019-06-14 08:47:58 +00:00
Riccardo Spagni	6ed1679bf8	prep for 0.14.1 release	2019-06-13 17:36:41 +02:00
luigi1111	3395de2e7f	Merge pull request #5633 `cfa88ac` Don't use -march=native (hyc) `e85bf46` Allow parallel make (hyc) `0ef8391` Delete redundant cppzmq dependency (hyc) `86591eb` Use 9 digit build IDs (hyc)	2019-06-12 14:50:01 -05:00
luigi1111	9f2882dbb7	Merge pull request #5631 `c27d961` [depends] update openssl to 1.0.2r (who-biz)	2019-06-12 14:45:40 -05:00
Howard Chu	86591ebf64	Use 9 digit build IDs	2019-06-12 16:15:07 +01:00
Howard Chu	0ef8391628	Delete redundant cppzmq dependency	2019-06-12 10:21:19 +01:00
Howard Chu	e85bf46641	Allow parallel make	2019-06-12 09:10:37 +01:00
Howard Chu	cfa88acb2b	Don't use -march=native	2019-06-12 09:10:29 +01:00
luigi1111	538fae4ec2	Merge pull request #5614 `4cff925` p2p: fix GCC 9.1 crash (monermooo-monero) `f47488c` Fix GCC 9.1 build warnings (moneromooo-monero) `ce13a98` cmake: do not use -mmitigate-rop on GCC >= 9.1 (moneromooo-monero)	2019-06-11 17:22:11 -05:00
luigi1111	0c62e7b15f	Merge pull request #5622 `b0a04f7` epee: fix SSL autodetect on reconnection (xiphon)	2019-06-11 17:05:17 -05:00
luigi1111	24806b5035	Merge pull request #5620 `117f950` miner: fix double free of thread attributes (ston1th)	2019-06-11 17:02:51 -05:00
luigi1111	0a1731aa7c	Merge pull request #5617 `6375111` miniupnpc: update to build on BSD (moneromooo-monero)	2019-06-11 17:01:14 -05:00
ston1th	117f9501d8	miner: fix double free of thread attributes issue: #5568	2019-06-09 12:29:03 +02:00
moneromooo-monero	ce13a98239	cmake: do not use -mmitigate-rop on GCC >= 9.1 It was removed, but it still accepted by the compiler, which warns for every file	2019-06-09 09:40:55 +00:00
moneromooo-monero	f47488c734	Fix GCC 9.1 build warnings GCC wants operator= aand copy ctor to be both defined, or neither	2019-06-09 09:39:34 +00:00
moneromooo-monero	6375111fa4	miniupnpc: update to build on BSD	2019-06-08 18:37:42 +00:00
moneromooo-monero	4cff9257e0	p2p: fix GCC 9.1 crash	2019-06-08 17:53:13 +00:00
Riccardo Spagni	256f8d8b66	Merge pull request #5584 `eeebad66` functional_tests: fix python3 compatibility (moneromooo-monero)	2019-06-01 20:32:39 +02:00
Riccardo Spagni	9a2883266c	Merge pull request #5578 `b7a96a08` core: update pruning if using --prune-blockchain on a pruned blockchain (moneromooo-monero)	2019-06-01 20:32:18 +02:00
Riccardo Spagni	3f2c82326e	Merge pull request #5572 `a663ccba` blockchain: do not try to pop blocks down to the genesis block (moneromooo-monero) `8f2a99d8` core: do not commit half constructed batch db txn (moneromooo-monero)	2019-06-01 20:31:55 +02:00
Riccardo Spagni	8b9920f0af	Merge pull request #5551 `de6cfacc` refresh and update translations for new release (erciccione)	2019-06-01 20:31:32 +02:00
Riccardo Spagni	162229286f	Merge pull request #5565 `4456a4b9` Fix allow any cert mode in wallet rpc when configured over rpc (Lee Clagett) `fafc5c36` Add ssl_options support to monerod's rpc mode. (Lee Clagett) `ce73cc3a` Fix configuration bug; wallet2 --daemon-ssl-allow-any-cert now works. (Lee Clagett)	2019-06-01 20:31:13 +02:00
Riccardo Spagni	2f5efc7f59	Merge pull request #5562 `13864702` functional_tests: fix rare get_output_distribution failure (moneromooo-monero)	2019-06-01 20:30:54 +02:00
Riccardo Spagni	0565fe21ec	Merge pull request #5563 `205a0ba1` unit_tests: make the density test a bit less stringent (moneromooo-monero)	2019-06-01 20:30:38 +02:00
Riccardo Spagni	ff5b30864a	Merge pull request #5564 `b6830db2` Fix #5553 (Howard Chu)	2019-06-01 20:30:01 +02:00
moneromooo-monero	eeebad6630	functional_tests: fix python3 compatibility Also add missing bans test to the default tests	2019-05-29 12:00:33 +00:00
moneromooo-monero	b7a96a0874	core: update pruning if using --prune-blockchain on a pruned blockchain Avoids a massive amount of spurious warnings if the last update before the daemon exited was a while ago and the daemon was syncing	2019-05-28 09:12:29 +00:00
moneromooo-monero	a663ccba71	blockchain: do not try to pop blocks down to the genesis block	2019-05-26 17:11:37 +00:00
moneromooo-monero	8f2a99d8ab	core: do not commit half constructed batch db txn	2019-05-25 16:25:10 +00:00
moneromooo-monero	13864702f1	functional_tests: fix rare get_output_distribution failure When the wallet auto refreshes after mining the last two blocks but before popping them, it will then try to use outputs which are not unlocked yet. This is really a wallet problem, which will be fixed later.	2019-05-22 11:24:08 +00:00
moneromooo-monero	205a0ba101	unit_tests: make the density test a bit less stringent It's an inherently random test	2019-05-22 11:23:20 +00:00
Lee Clagett	4456a4b9b3	Fix allow any cert mode in wallet rpc when configured over rpc	2019-05-21 16:17:50 +00:00
Lee Clagett	fafc5c3692	Add ssl_options support to monerod's rpc mode.	2019-05-21 16:17:34 +00:00
Lee Clagett	ce73cc3add	Fix configuration bug; wallet2 --daemon-ssl-allow-any-cert now works.	2019-05-21 16:17:13 +00:00
Howard Chu	b6830db2d4	Fix #5553 Make sure the tip hash still matches the cached block	2019-05-19 03:02:33 +01:00
who-biz	c27d96129e	[depends] update openssl to 1.0.2r - This addresses https://www.openssl.org/news/secadv/20190226.txt (CVE: 2019-1559) which impacted all versions of openssl-1.0. Note that this does not address CVE-2019-1543 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-1543), which impacts all versions of openssl 1.1 through 1.1.0j and 1.1.1b. The above (1.1) is patched in openssl, where it was marked as low severity. Similar issues possibly present in monero, should be looked into w.r.t. CVE-2019-1543.	2019-05-18 02:18:55 -04:00
erciccione	de6cfacc02	refresh and update translations for new release	2019-05-17 14:13:10 +02:00