forked from continuwuation/rocksdb
86bb0c0d1b
Summary: A follow-up to https://github.com/facebook/rocksdb/issues/13904 which was incomplete in updating CI jobs to support C++20 because the C++20 usage was only in tests. Here we add subtle C++20 usage in the public API ("using enum" feature in db.h) to force the issue. A lot of the work for this PR was in updating the Ubuntu22 docker image, for earlier compiler/runtime versions supporting C++20, and generating a new Ubuntu24 docker image, for later compiler/runtime versions. The Ubuntu22 image needed to be updated because there are incompatibilities with clang-13 + c++20 + libstdc++ for gcc 11, seen on these examples ``` #include <chrono> int main(int argc, char *argv[]) { std::chrono::microseconds d = {}; return 0; } ``` and ``` #include <coroutine> int main() { return 0; } ``` The second was causing recurring failures in build-linux-clang-13-asan-ubsan-with-folly, now fixed. So we have to install clang's libc++ to compile with clang-13. I haven't been able to get this to work with some of the libraries like benchmark, glog, and/or gflags, but I'm able to compile core RocksDB with clang-13. On this docker image, an extra compiler parameter is needed to compile with gcc and glog because it's built from source perhaps not perfectly, because the ubuntu package transitively conflicts with libc++. The Ubuntu24 image seems to be low-drama and generally work for testing out newer compiler versions. The mingw build uses Ubuntu24 because the mingw package on Ubuntu22 uses a gcc version that is too old. And the mass of other code changes are trying to work around new warnings, mostly from clang-analyze, which I upgraded to clang-18 in CI. Pull Request resolved: https://github.com/facebook/rocksdb/pull/13915 Test Plan: CI, including temporarily including the nightly jobs in the PR jobs in earlier revisions to test and stabilize Reviewed By: archang19 Differential Revision: D81933067 Pulled By: pdillinger fbshipit-source-id: 7e33823006a79d5f3cf5bc1d625f0a3c08a7d74c
88 lines
4.3 KiB
Docker
88 lines
4.3 KiB
Docker
# INSTRUCTIONS:
|
|
# I was not able to build docker images on an isolated devserver because of
|
|
# issues with proxy internet access. Use a public cloud or other Linux system.
|
|
# (I used a Debian system after installing docker features, adding my user to
|
|
# the docker and docker-registry groups, and logging out and back in to pick
|
|
# those up.)
|
|
#
|
|
# Follow https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic
|
|
# to login with your GitHub credentials, as in
|
|
#
|
|
# $ docker login ghcr.io -u pdillinger
|
|
#
|
|
# and paste the limited-purpose GitHub token into the terminal.
|
|
#
|
|
# Then in the build_tools/ubuntu22_image directory, (bump minor version for
|
|
# random docker file updates, major version tracks Ubuntu release)
|
|
#
|
|
# $ docker build -t ghcr.io/facebook/rocksdb_ubuntu:22.0
|
|
# $ docker push ghcr.io/facebook/rocksdb_ubuntu:22.0
|
|
#
|
|
# Might need to change visibility to public through
|
|
# https://github.com/orgs/facebook/packages/container/rocksdb_ubuntu/settings
|
|
# or similar.
|
|
|
|
# from official ubuntu 22.04
|
|
FROM ubuntu:22.04
|
|
# update system
|
|
RUN apt-get update
|
|
RUN apt-get upgrade -y
|
|
# install basic tools
|
|
RUN apt-get install -y vim wget curl
|
|
# install tzdata noninteractive
|
|
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata
|
|
# install git and default compilers
|
|
RUN apt-get install -y git gcc g++ clang clang-tools
|
|
# install basic package
|
|
RUN apt-get install -y lsb-release software-properties-common gnupg
|
|
# install gflags, tbb
|
|
RUN apt-get install -y libgflags-dev libtbb-dev
|
|
# install compression libs
|
|
RUN apt-get install -y libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev
|
|
# install cmake
|
|
RUN apt-get install -y cmake
|
|
RUN apt-get install -y libssl-dev
|
|
# install clang-13
|
|
WORKDIR /root
|
|
RUN wget https://apt.llvm.org/llvm.sh
|
|
RUN chmod +x llvm.sh
|
|
RUN ./llvm.sh 13 all
|
|
# There are incompatibilities between clang with -std=c++20 and libstdc++
|
|
# provided by gcc, so we have to compile with clang-13 using -stdlib=libc++
|
|
# and only one version of libc++ can be installed on the system at one time.
|
|
# So to avoid confusion we remove unusable clang-14 also.
|
|
RUN apt-get install libc++-13-dev libc++abi-13-dev
|
|
RUN apt-get purge -y clang-14 && apt-get autoremove -y
|
|
|
|
# install gcc-10 and more, default is 11
|
|
RUN apt-get install -y gcc-10 g++-10
|
|
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test
|
|
RUN apt-get install -y gcc-13 g++-13
|
|
# install apt-get install -y valgrind
|
|
RUN apt-get install -y valgrind
|
|
# install folly depencencies
|
|
# Missing compatible libunwind: RUN apt-get install -y libgoogle-glog-dev
|
|
# So instead install from source. This currently requires compiling with
|
|
# -DGLOG_USE_GLOG_EXPORT
|
|
RUN wget https://github.com/google/glog/archive/refs/tags/v0.7.1.tar.gz && tar xzf v0.7.1.tar.gz && cd glog-0.7.1/ && cmake -S . -B build -G "Unix Makefiles" && cmake --build build && cmake --build build --target install && cd .. && rm -rf v0.7.1.tar.gz glog-0.7.1
|
|
# install openjdk 8
|
|
RUN apt-get install -y openjdk-8-jdk
|
|
ENV JAVA_HOME /usr/lib/jvm/java-1.8.0-openjdk-amd64
|
|
# install mingw
|
|
RUN apt-get install -y mingw-w64
|
|
|
|
# install gtest-parallel package
|
|
RUN git clone --single-branch --branch master --depth 1 https://github.com/google/gtest-parallel.git ~/gtest-parallel
|
|
ENV PATH $PATH:/root/gtest-parallel
|
|
|
|
# install libprotobuf for fuzzers test
|
|
RUN apt-get install -y ninja-build binutils liblzma-dev libz-dev pkg-config autoconf libtool
|
|
RUN git clone --branch v1.0 https://github.com/google/libprotobuf-mutator.git ~/libprotobuf-mutator && cd ~/libprotobuf-mutator && git checkout ffd86a32874e5c08a143019aad1aaf0907294c9f && mkdir build && cd build && cmake .. -GNinja -DCMAKE_C_COMPILER=clang-13 -DCMAKE_CXX_COMPILER=clang++-13 -DCMAKE_BUILD_TYPE=Release -DLIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON && ninja && ninja install
|
|
ENV PKG_CONFIG_PATH /usr/local/OFF/:/root/libprotobuf-mutator/build/external.protobuf/lib/pkgconfig/
|
|
ENV PROTOC_BIN /root/libprotobuf-mutator/build/external.protobuf/bin/protoc
|
|
|
|
# install the latest google benchmark
|
|
RUN git clone --depth 1 --branch v1.7.0 https://github.com/google/benchmark.git ~/benchmark && cd ~/benchmark && mkdir build && cd build && cmake .. -GNinja -DCMAKE_BUILD_TYPE=Release -DBENCHMARK_ENABLE_GTEST_TESTS=0 && ninja && ninja install && cd ~ && rm -rf /root/benchmark
|
|
|
|
# clean up
|
|
RUN rm -rf /var/lib/apt/lists/*
|