Running mongodb unit and dbtests

MongoDB Server Development
#1

Hi,
I was trying to build mongodb source code and run unit and integration tests. But looks like the documentation on mongodb wiki is older? https://github.com/mongodb/mongo/wiki/Test-The-Mongodb-Server
I face issues running the the resmoke script
python buildscripts/resmoke.py --help
gives error as it needs evergreen module.
Is there any script or better yet, a docker image with all the dependencies, to build and run mongo tests?

Thanks,
Unmesh

#2

Hi Unmesh,

Thank you for the question. The testing documentation wiki page has been updated with additional setup steps. See the updated section on resmoke.py here.

Robert

1 Like
#3

I had to run following to install dev-requirements, which installed the correct versions of evergreen and other dependencies

python3 -m pip install -r etc/pip/dev-requirements.txt

For building unittests (and generating build/unittests.txt), whats the target?

#4

It seems to be install-unittests and not unittests.
Is there a quick developer build option? install-unittests seems to be taking a lot of disk space.

#5

Try running ./buildscripts/scons.py --ninja generate-ninja to generate a ninja build file. Then if you run ninja +<string> it will give you a list of unit test targets starting with . This requires the ninja build tool to be installed.

#6

Building with Ninja is unlikely to be much faster for a full build than building with SCons unless you have a compute cluster and can drive huge amounts of parallel compile through it. Ninja will be faster to decide what must be rebuilt for an incremental build after making source changes, but it still can’t drive more compile jobs than you have accessible cores. However, if disk space is a concern, you can build with --link-model=dynamic --install-action=hardlink. Note that the resulting binaries are not production quality, but that should be fine because it appears that your goal is to run the various tests.

#7

Thanks. I am interested in running tests. Mostly to understand some of the implementation aspects for the work I am doing to documents patterns of distributed systems at https://martinfowler.com/articles/patterns-of-distributed-systems/
Docker image build with following Dockerfile builds the source code perfectly fine. If I use Ubuntu20.04 base image instead of Ubuntu18.04, I face issues. But thats fine, I can continue to use Ubuntu18.04
Having this kind of Docker images to build will be useful for new developers. Something that scylladb folks do.
Just curious what is the machine configurations developers use to work with mongodb source code?
Thinkpad P1 gen2 does not seem to be good enough. Any recommendations will be helpful.

FROM ubuntu:18.04
RUN apt update
RUN apt install software-properties-common -y
RUN apt update
RUN apt install git -y
RUN apt install python3.8 -y
RUN apt install python3-pip -y
RUN apt install libpython3.8-dev -y
RUN ln -S -f /usr/bin/python3.8 /usr/bin/python3
RUN apt install g+±8 -y
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 800 --slave /usr/bin/g++ g++ /usr/bin/g+±8
RUN apt-get install libssl-dev -y
RUN apt-get install -y liblzma-dev
RUN apt-get install libcurl4-openssl-dev -y

Copy and setup python dependencies as an optimization.
COPY pip /pip
RUN python3 -m pip install -r /pip/dev-requirements.txt

VOLUME build
CMD ["/bin/bash"]

#8

With the docker container mentioned above and using link-model=dynamic --install-action=hardlink, I could run tests.
Is there a way to run individual tests?

#9

Yes, just run the relevant test binary out of the build/install/bin directory after building it with SCons. You can also just build the specific test you are interested in by naming it as the target to scons: scons ... build/install/bin/foo_test will build the foo_test binary, which you can then run from the command line.

#10

Replying to your request about developer machines, it varies, but I would consider something like the highest end MacBook Pro to be the bare minimum. Most developers use linux workstations with at least 16 cores for database development, and often quite a bit more than that. You really want a workstation class system to be able to work with the codebase effectively.

#11

Thank you. I was able to build using the docker image I built and also run specific unit tests.
Just so that this does’nt get lost and might be useful for someone new wanting to build mongo db source code, I have created a pull request with instructions to build and run tests using docker container