Skip to content

Instantly share code, notes, and snippets.

@smnorris

smnorris/Dockerfile

Last active February 11, 2022 20:08
Show Gist options
  • Save smnorris/01cf5147d73cec1d05a9ec149b5f264e to your computer and use it in GitHub Desktop.
Save smnorris/01cf5147d73cec1d05a9ec149b5f264e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Dockerfile
##
# osgeo/gdal:ubuntu-small
# This file is available at the option of the licensee under:
# Public domain
# or licensed under MIT (LICENSE.TXT) Copyright 2019 Even Rouault <[email protected]>
ARG PROJ_INSTALL_PREFIX=/usr/local
ARG BASE_IMAGE=ubuntu:20.04
ARG TARGET_BASE_IMAGE=ubuntu:20.04
FROM $BASE_IMAGE as builder
# Derived from osgeo/proj by Howard Butler <[email protected]>
LABEL maintainer="Even Rouault <[email protected]>"
ARG TARGET_ARCH=
RUN echo ${TARGET_ARCH}
COPY ./bh-set-envvars.sh /buildscripts/bh-set-envvars.sh
RUN . /buildscripts/bh-set-envvars.sh \
&& if test "${TARGET_ARCH}" != ""; then \
rm -f /etc/apt/sources.list \
&& echo "deb [arch=amd64] http://us.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse" >> /etc/apt/sources.list \
&& echo "deb [arch=amd64] http://us.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse" >> /etc/apt/sources.list \
&& echo "deb [arch=amd64] http://us.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse" >> /etc/apt/sources.list \
&& echo "deb [arch=amd64] http://security.ubuntu.com/ubuntu focal-security main restricted universe multiverse" >> /etc/apt/sources.list \
&& echo "deb [arch=${TARGET_ARCH}] http://ports.ubuntu.com/ubuntu-ports/ focal main restricted universe multiverse" >> /etc/apt/sources.list \
&& echo "deb [arch=${TARGET_ARCH}] http://ports.ubuntu.com/ubuntu-ports/ focal-updates main restricted universe multiverse" >> /etc/apt/sources.list \
&& echo "deb [arch=${TARGET_ARCH}] http://ports.ubuntu.com/ubuntu-ports/ focal-security main restricted universe multiverse" >> /etc/apt/sources.list \
&& dpkg --add-architecture ${TARGET_ARCH} \
&& apt-get update -y \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y g++-9-${GCC_ARCH}-linux-gnu \
&& ln -s ${GCC_ARCH}-linux-gnu-gcc-9 /usr/bin/${GCC_ARCH}-linux-gnu-gcc \
&& ln -s ${GCC_ARCH}-linux-gnu-g++-9 /usr/bin/${GCC_ARCH}-linux-gnu-g++; \
fi
# Setup build env for PROJ
USER root
RUN . /buildscripts/bh-set-envvars.sh \
&& apt-get update -y \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --fix-missing --no-install-recommends \
build-essential ca-certificates \
git make cmake wget unzip libtool automake \
zlib1g-dev${APT_ARCH_SUFFIX} libsqlite3-dev${APT_ARCH_SUFFIX} pkg-config sqlite3 libcurl4-gnutls-dev${APT_ARCH_SUFFIX} \
libtiff5-dev${APT_ARCH_SUFFIX}
# Setup build env for GDAL
RUN . /buildscripts/bh-set-envvars.sh \
&& apt-get update -y \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --fix-missing --no-install-recommends \
python3-dev${APT_ARCH_SUFFIX} python3-numpy${APT_ARCH_SUFFIX} python3-setuptools${APT_ARCH_SUFFIX} \
libjpeg-dev${APT_ARCH_SUFFIX} libgeos-dev${APT_ARCH_SUFFIX} \
libexpat-dev${APT_ARCH_SUFFIX} libxerces-c-dev${APT_ARCH_SUFFIX} \
libwebp-dev${APT_ARCH_SUFFIX} libpng-dev${APT_ARCH_SUFFIX} \
libzstd-dev${APT_ARCH_SUFFIX} bash zip curl \
libpq-dev${APT_ARCH_SUFFIX} libssl-dev${APT_ARCH_SUFFIX} libopenjp2-7-dev${APT_ARCH_SUFFIX} \
autoconf automake sqlite3 bash-completion
# Build openjpeg
ARG OPENJPEG_VERSION=
RUN . /buildscripts/bh-set-envvars.sh \
&& if test "${OPENJPEG_VERSION}" != ""; then ( \
wget -q https://github.com/uclouvain/openjpeg/archive/v${OPENJPEG_VERSION}.tar.gz \
&& tar xzf v${OPENJPEG_VERSION}.tar.gz \
&& rm -f v${OPENJPEG_VERSION}.tar.gz \
&& cd openjpeg-${OPENJPEG_VERSION} \
&& cmake . -DBUILD_SHARED_LIBS=ON -DBUILD_STATIC_LIBS=OFF -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr \
&& make -j$(nproc) \
&& make install \
&& mkdir -p /build_thirdparty/usr/lib \
&& cp -P /usr/lib/libopenjp2*.so* /build_thirdparty/usr/lib \
&& for i in /build_thirdparty/usr/lib/*; do strip -s $i 2>/dev/null || /bin/true; done \
&& cd .. \
&& rm -rf openjpeg-${OPENJPEG_VERSION} \
); fi
ARG PROJ_INSTALL_PREFIX
ARG PROJ_DATUMGRID_LATEST_LAST_MODIFIED
RUN \
mkdir -p /build_projgrids/${PROJ_INSTALL_PREFIX}/share/proj \
&& curl -LOs http://download.osgeo.org/proj/proj-datumgrid-latest.zip \
&& unzip -q -j -u -o proj-datumgrid-latest.zip -d /build_projgrids/${PROJ_INSTALL_PREFIX}/share/proj \
&& rm -f *.zip
RUN apt-get update -y \
&& apt-get install -y --fix-missing --no-install-recommends rsync ccache
ARG RSYNC_REMOTE
# Build PROJ
ARG PROJ_VERSION=master
RUN . /buildscripts/bh-set-envvars.sh \
&& mkdir proj \
&& wget -q https://github.com/OSGeo/PROJ/archive/${PROJ_VERSION}.tar.gz -O - \
| tar xz -C proj --strip-components=1 \
&& cd proj \
&& if test "${RSYNC_REMOTE:-}" != ""; then \
echo "Downloading cache..."; \
rsync -ra ${RSYNC_REMOTE}/proj/${GCC_ARCH}/ $HOME/; \
echo "Finished"; \
export CC="ccache ${GCC_ARCH}-linux-gnu-gcc"; \
export CXX="ccache ${GCC_ARCH}-linux-gnu-g++"; \
export PROJ_DB_CACHE_DIR="$HOME/.ccache"; \
ccache -M 100M; \
fi \
&& CFLAGS='-DPROJ_RENAME_SYMBOLS -O2' CXXFLAGS='-DPROJ_RENAME_SYMBOLS -DPROJ_INTERNAL_CPP_NAMESPACE -O2' \
cmake . \
-DBUILD_SHARED_LIBS=ON \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=${PROJ_INSTALL_PREFIX} \
-DBUILD_TESTING=OFF \
&& make -j$(nproc) \
&& make install DESTDIR="/build" \
&& if test "${RSYNC_REMOTE:-}" != ""; then \
ccache -s; \
echo "Uploading cache..."; \
rsync -ra --delete $HOME/.ccache ${RSYNC_REMOTE}/proj/${GCC_ARCH}/; \
echo "Finished"; \
rm -rf $HOME/.ccache; \
unset CC; \
unset CXX; \
fi \
&& cd .. \
&& rm -rf proj \
&& PROJ_SO=$(readlink -f /build${PROJ_INSTALL_PREFIX}/lib/libproj.so | awk 'BEGIN {FS="libproj.so."} {print $2}') \
&& PROJ_SO_FIRST=$(echo $PROJ_SO | awk 'BEGIN {FS="."} {print $1}') \
&& mv /build${PROJ_INSTALL_PREFIX}/lib/libproj.so.${PROJ_SO} /build${PROJ_INSTALL_PREFIX}/lib/libinternalproj.so.${PROJ_SO} \
&& ln -s libinternalproj.so.${PROJ_SO} /build${PROJ_INSTALL_PREFIX}/lib/libinternalproj.so.${PROJ_SO_FIRST} \
&& ln -s libinternalproj.so.${PROJ_SO} /build${PROJ_INSTALL_PREFIX}/lib/libinternalproj.so \
&& rm /build${PROJ_INSTALL_PREFIX}/lib/libproj.* \
&& ln -s libinternalproj.so.${PROJ_SO} /build${PROJ_INSTALL_PREFIX}/lib/libproj.so.${PROJ_SO_FIRST} \
&& ${GCC_ARCH}-linux-gnu-strip -s /build${PROJ_INSTALL_PREFIX}/lib/libinternalproj.so.${PROJ_SO} \
&& for i in /build${PROJ_INSTALL_PREFIX}/bin/*; do ${GCC_ARCH}-linux-gnu-strip -s $i 2>/dev/null || /bin/true; done
ARG WITH_FILEGDB=
RUN . /buildscripts/bh-set-envvars.sh \
&& if echo "$WITH_FILEGDB" | grep -Eiq "^(y(es)?|1|true)$" ; then ( \
wget -q https://github.com/Esri/file-geodatabase-api/raw/master/FileGDB_API_1.5.2/FileGDB_API-RHEL7-64gcc83.tar.gz \
&& tar -xzf FileGDB_API-RHEL7-64gcc83.tar.gz \
&& chown -R root:root FileGDB_API-RHEL7-64gcc83 \
&& mv FileGDB_API-RHEL7-64gcc83 /usr/local/FileGDB_API \
&& rm -rf /usr/local/FileGDB_API/lib/libstdc++* \
&& mkdir -p /build_thirdparty/usr/lib \
&& cp /usr/local/FileGDB_API/lib/* /build_thirdparty/usr/lib \
&& cp /usr/local/FileGDB_API/include/* /usr/include \
&& rm -rf FileGDB_API-RHEL7-64gcc83.tar.gz \
) ; fi
# Build GDAL
ARG GDAL_VERSION=master
ARG GDAL_RELEASE_DATE
ARG GDAL_BUILD_IS_RELEASE
ARG GDAL_REPOSITORY=OSGeo/gdal
RUN . /buildscripts/bh-set-envvars.sh \
&& if test "${GDAL_VERSION}" = "master"; then \
export GDAL_VERSION=$(curl -Ls https://api.github.com/repos/${GDAL_REPOSITORY}/commits/HEAD -H "Accept: application/vnd.github.VERSION.sha"); \
export GDAL_RELEASE_DATE=$(date "+%Y%m%d"); \
fi \
&& if test "x${GDAL_BUILD_IS_RELEASE:-}" = "x"; then \
export GDAL_SHA1SUM=${GDAL_VERSION}; \
fi \
&& mkdir gdal \
&& wget -q https://github.com/${GDAL_REPOSITORY}/archive/${GDAL_VERSION}.tar.gz -O - \
| tar xz -C gdal --strip-components=1 \
&& cd gdal \
&& ./autogen.sh \
&& if test "${RSYNC_REMOTE:-}" != ""; then \
echo "Downloading cache..."; \
rsync -ra ${RSYNC_REMOTE}/gdal/${GCC_ARCH}/ $HOME/; \
echo "Finished"; \
# Little trick to avoid issues with Python bindings
printf "#!/bin/sh\nccache %s-linux-gnu-gcc \$*" "${GCC_ARCH}" > ccache_gcc.sh; \
chmod +x ccache_gcc.sh; \
printf "#!/bin/sh\nccache %s-linux-gnu-g++ \$*" "${GCC_ARCH}" > ccache_g++.sh; \
chmod +x ccache_g++.sh; \
export CC=$PWD/ccache_gcc.sh; \
export CXX=$PWD/ccache_g++.sh; \
ccache -M 1G; \
fi \
&& export LD_LIBRARY_PATH=/usr/local/FileGDB_API/lib \
&& LDFLAGS="-L/build${PROJ_INSTALL_PREFIX}/lib -linternalproj" \
./configure --prefix=/usr --sysconfdir=/etc "${WITH_HOST}" --without-libtool \
--with-fgdb=/usr/local/FileGDB_API \
--with-hide-internal-symbols \
--with-jpeg12 \
--with-python \
--with-webp --with-proj=/build${PROJ_INSTALL_PREFIX} \
--with-libtiff=internal --with-rename-internal-libtiff-symbols \
--with-geotiff=internal --with-rename-internal-libgeotiff-symbols \
&& make -j$(nproc) \
&& make install DESTDIR="/build" \
&& if test "${RSYNC_REMOTE:-}" != ""; then \
ccache -s; \
echo "Uploading cache..."; \
rsync -ra --delete $HOME/.ccache ${RSYNC_REMOTE}/gdal/${GCC_ARCH}/; \
echo "Finished"; \
rm -rf $HOME/.ccache; \
unset CC; \
unset CXX; \
fi \
&& cd .. \
&& rm -rf gdal \
&& mkdir -p /build_gdal_python/usr/lib \
&& mkdir -p /build_gdal_python/usr/bin \
&& mkdir -p /build_gdal_version_changing/usr/include \
&& mv /build/usr/lib/python* /build_gdal_python/usr/lib \
&& mv /build/usr/lib /build_gdal_version_changing/usr \
&& mv /build/usr/include/gdal_version.h /build_gdal_version_changing/usr/include \
&& mv /build/usr/bin/*.py /build_gdal_python/usr/bin \
&& mv /build/usr/bin /build_gdal_version_changing/usr \
&& for i in /build_gdal_version_changing/usr/lib/*; do ${GCC_ARCH}-linux-gnu-strip -s $i 2>/dev/null || /bin/true; done \
&& for i in /build_gdal_python/usr/lib/python3/dist-packages/osgeo/*.so; do ${GCC_ARCH}-linux-gnu-strip -s $i 2>/dev/null || /bin/true; done \
&& for i in /build_gdal_version_changing/usr/bin/*; do ${GCC_ARCH}-linux-gnu-strip -s $i 2>/dev/null || /bin/true; done
# Build final image
FROM $TARGET_BASE_IMAGE as runner
USER root
RUN date
# PROJ dependencies
RUN apt-get update; \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
libsqlite3-0 libtiff5 libcurl4 \
curl unzip ca-certificates
# GDAL dependencies
RUN apt-get update -y; \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
python3-numpy libpython3.8 \
libjpeg-turbo8 libgeos-3.8.0 libgeos-c1v5 \
libexpat1 \
libxerces-c3.2 \
libwebp6 libpng16-16 \
libzstd1 bash libpq5 libssl1.1 libopenjp2-7
RUN apt-get update; \
DEBIAN_FRONTEND=noninteractive apt-get install -y python-is-python3
# Order layers starting with less frequently varying ones
# Only used for custom libopenjp2
COPY --from=builder /build_thirdparty/usr/ /usr/
COPY --from=builder /build_projgrids/usr/ /usr/
ARG PROJ_INSTALL_PREFIX
COPY --from=builder /build${PROJ_INSTALL_PREFIX}/share/proj/ ${PROJ_INSTALL_PREFIX}/share/proj/
COPY --from=builder /build${PROJ_INSTALL_PREFIX}/include/ ${PROJ_INSTALL_PREFIX}/include/
COPY --from=builder /build${PROJ_INSTALL_PREFIX}/bin/ ${PROJ_INSTALL_PREFIX}/bin/
COPY --from=builder /build${PROJ_INSTALL_PREFIX}/lib/ ${PROJ_INSTALL_PREFIX}/lib/
COPY --from=builder /build/usr/share/gdal/ /usr/share/gdal/
COPY --from=builder /build/usr/include/ /usr/include/
COPY --from=builder /build_gdal_python/usr/ /usr/
COPY --from=builder /build_gdal_version_changing/usr/ /usr/
RUN ldconfig
Raw
macOS_gdal_Docker.md

GDAL with FGDB API, Jan2022

TLDR; use Docker

I'm sure this can be done with a package manager rather than compiling GDAL, but Docker makes it easy.

Requirements

Docker

Installation

$ git clone https://github.com/OSGeo/gdal.git
$ cd gdal/docker/ubuntu-full
$ docker build --build-arg WITH_FILEGDB=yes -t osgeo/gdal:fgdb

Grab a coffee, this takes ~1hr 20min...

Usage

$ docker run --rm osgeo/gdal:fgdb ogrinfo --formats

Optional

Make it the default for ogr2ogr:

$ alias ogr2ogr='docker run --rm osgeo/gdal:fgdb ogr2ogr'
$ ogr2ogr --formats | grep GDB
  OpenFileGDB -vector- (rov): ESRI FileGDB
  FileGDB -vector- (rw+): ESRI FileGDB

UPDATE FEB 2022 - M1 mac with macOS 12.1

The ubuntu-full Dockerfile does not build successfully on my M1 mac (error with Java). Edit ubuntu-small Dockerfile to add fgdb support.

Then:

$ docker build --platform linux/amd64 --build-arg WITH_FILEGDB=yes -t osgeo/gdal:fgdb .
$ docker run --platform linux/amd64 --rm osgeo/gdal:fgdb ogrinfo --formats | grep ESRI
  ESRI Shapefile -vector- (rw+v): ESRI Shapefile
  ESRIJSON -vector- (rov): ESRIJSON
  OpenFileGDB -vector- (rov): ESRI FileGDB
  FileGDB -vector- (rw+): ESRI FileGDB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment