Seeking hardening flag / blhc expoert

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Seeking hardening flag / blhc expoert

Otto Kekäläinen-4
Hello!

Is there any hardening flag / cmake expert around who could help me
get the hardening flags perfect in MariaDB 10.3?

Current state of build logs issues:
https://qa.debian.org/bls/packages/m/mariadb-10.3.html

The blhc tool currently outputs this:

$ blhc --debian --line-numbers --color ${WORKING_DIR}/*.build || [ $? -eq 1 ]
9962:CPPFLAGS missing (-D_FORTIFY_SOURCE=2):
/usr/lib/ccache/x86_64-linux-gnu-g++
-I/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy
-std=c++11 -g -O2 -fdebug-prefix-map=/tmp/building/package=.
-fstack-protector-strong -Wformat -Werror=format-security -pie -fPIC
-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4
-fno-rtti -Wno-shadow -Wno-implicit-fallthrough -std=c++11
-Wno-missing-field-initializers -Wno-missing-field-initializers
-Wstrict-null-sentinel -Winit-self -Wswitch -Wtrampolines -Wlogical-op
-Wno-error=missing-format-attribute -Wno-error=maybe-uninitialized
-fno-rtti -fno-exceptions -Wno-error=nonnull-compare -Wpacked
-fno-omit-frame-pointer -Wno-error=strict-overflow -fexceptions
-Wextra -Wno-missing-noreturn -Wmissing-declarations -Wpointer-arith
-Wcast-align -O2 -g -DNDEBUG -fPIC -Wno-sign-compare
-Wno-unused-function -Wno-unused-parameter -fvisibility=hidden -fPIC
-o CMakeFiles/snappy.dir/snappy.cc.o -c
/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy/snappy.cc
9964:CPPFLAGS missing (-D_FORTIFY_SOURCE=2):
/usr/lib/ccache/x86_64-linux-gnu-g++
-I/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy
-std=c++11 -g -O2 -fdebug-prefix-map=/tmp/building/package=.
-fstack-protector-strong -Wformat -Werror=format-security -pie -fPIC
-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4
-fno-rtti -Wno-shadow -Wno-implicit-fallthrough -std=c++11
-Wno-missing-field-initializers -Wno-missing-field-initializers
-Wstrict-null-sentinel -Winit-self -Wswitch -Wtrampolines -Wlogical-op
-Wno-error=missing-format-attribute -Wno-error=maybe-uninitialized
-fno-rtti -fno-exceptions -Wno-error=nonnull-compare -Wpacked
-fno-omit-frame-pointer -Wno-error=strict-overflow -fexceptions
-Wextra -Wno-missing-noreturn -Wmissing-declarations -Wpointer-arith
-Wcast-align -O2 -g -DNDEBUG -fPIC -Wno-sign-compare
-Wno-unused-function -Wno-unused-parameter -fvisibility=hidden -fPIC
-o CMakeFiles/snappy.dir/snappy-c.cc.o -c
/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy/snappy-c.cc
9966:CPPFLAGS missing (-D_FORTIFY_SOURCE=2):
/usr/lib/ccache/x86_64-linux-gnu-g++
-I/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy
-std=c++11 -g -O2 -fdebug-prefix-map=/tmp/building/package=.
-fstack-protector-strong -Wformat -Werror=format-security -pie -fPIC
-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4
-fno-rtti -Wno-shadow -Wno-implicit-fallthrough -std=c++11
-Wno-missing-field-initializers -Wno-missing-field-initializers
-Wstrict-null-sentinel -Winit-self -Wswitch -Wtrampolines -Wlogical-op
-Wno-error=missing-format-attribute -Wno-error=maybe-uninitialized
-fno-rtti -fno-exceptions -Wno-error=nonnull-compare -Wpacked
-fno-omit-frame-pointer -Wno-error=strict-overflow -fexceptions
-Wextra -Wno-missing-noreturn -Wmissing-declarations -Wpointer-arith
-Wcast-align -O2 -g -DNDEBUG -fPIC -Wno-sign-compare
-Wno-unused-function -Wno-unused-parameter -fvisibility=hidden -fPIC
-o CMakeFiles/snappy.dir/snappy-sinksource.cc.o -c
/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy/snappy-sinksource.cc
9968:CPPFLAGS missing (-D_FORTIFY_SOURCE=2):
/usr/lib/ccache/x86_64-linux-gnu-g++
-I/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy
-std=c++11 -g -O2 -fdebug-prefix-map=/tmp/building/package=.
-fstack-protector-strong -Wformat -Werror=format-security -pie -fPIC
-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4
-fno-rtti -Wno-shadow -Wno-implicit-fallthrough -std=c++11
-Wno-missing-field-initializers -Wno-missing-field-initializers
-Wstrict-null-sentinel -Winit-self -Wswitch -Wtrampolines -Wlogical-op
-Wno-error=missing-format-attribute -Wno-error=maybe-uninitialized
-fno-rtti -fno-exceptions -Wno-error=nonnull-compare -Wpacked
-fno-omit-frame-pointer -Wno-error=strict-overflow -fexceptions
-Wextra -Wno-missing-noreturn -Wmissing-declarations -Wpointer-arith
-Wcast-align -O2 -g -DNDEBUG -fPIC -Wno-sign-compare
-Wno-unused-function -Wno-unused-parameter -fvisibility=hidden -fPIC
-o CMakeFiles/snappy.dir/snappy-stubs-internal.cc.o -c
/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy/snappy-stubs-internal.cc

Full log at:
https://salsa.debian.org/mariadb-team/mariadb-10.3/-/jobs/153422

d/rules:
https://salsa.debian.org/mariadb-team/mariadb-10.3/blob/master/debian/rules

Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Andrey Rahmatullin-3
On Fri, Apr 05, 2019 at 09:07:01PM +0300, Otto Kekäläinen wrote:
> Is there any hardening flag / cmake expert around who could help me
> get the hardening flags perfect in MariaDB 10.3?
Start with https://wiki.debian.org/Hardening#Notes_for_packages_using_CMake

> d/rules:
> https://salsa.debian.org/mariadb-team/mariadb-10.3/blob/master/debian/rules
One of the problems is using $(MAKE) instead of dh_auto_build and so on.
There are other problems in this file.

--
WBR, wRAR

signature.asc (911 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Otto Kekäläinen-4
Hello!

> > Is there any hardening flag / cmake expert around who could help me
> > get the hardening flags perfect in MariaDB 10.3?
> Start with https://wiki.debian.org/Hardening#Notes_for_packages_using_CMake

I've read this section many times over but I don't get it. A
workaround is presented but since we are on a new debhelper it is
advised not to be used. It suggests using
/usr/share/dpkg/buildflags.mk but since we already call default.mk the
buildflags.mk should be included. There are some variables set, but
since the cmake command does not include them, changes in them does
not have an effect. There is no explanation about that flags do what
and which are the relevant ones, so blindly just defining everything
does not seem like a savvy solution.

I would appreciate if you can pinpoint what is the missing flag
exactly and what is now not passed to cmake correctly..

> > d/rules:
> > https://salsa.debian.org/mariadb-team/mariadb-10.3/blob/master/debian/rules
> One of the problems is using $(MAKE) instead of dh_auto_build and so on.
> There are other problems in this file.

Since the build command is constructed in the
override_dh_auto_configure stanza this is the only way I am aware that
I can pass it on to dh_auto_build. I am happy to try out alternative
ways if you have concrete suggestions on how to refactor the d/rules
file

Thanks for pointers and help!

Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Sven Hartge-5
Otto Kekäläinen <[hidden email]> wrote:

> I've read this section many times over but I don't get it. A
> workaround is presented but since we are on a new debhelper it is
> advised not to be used. It suggests using
> /usr/share/dpkg/buildflags.mk but since we already call default.mk the
> buildflags.mk should be included. There are some variables set, but
> since the cmake command does not include them, changes in them does
> not have an effect. There is no explanation about that flags do what
> and which are the relevant ones, so blindly just defining everything
> does not seem like a savvy solution.

CMake is a bit "special" in that regard. To get the right hardening
flags to work for some parts of Bacula, we had to include the following
patch to kind-of brute force the flags:
https://salsa.debian.org/bacula-team/bacula/blob/master/debian/patches/debian/enable-hardening-for-qmake

,----
| --- a/src/qt-console/bat.pro.in
| +++ b/src/qt-console/bat.pro.in
| @@ -187,3 +187,9 @@
|  QMAKE_EXTRA_TARGETS += depend
|  
|  TRANSLATIONS += ts/bat_fr.ts ts/bat_de.ts
| +
| +### See https://wiki.debian.org/Hardening#Notes_for_packages_using_QMake
| +QMAKE_CPPFLAGS *= $(shell dpkg-buildflags --get CPPFLAGS)
| +QMAKE_CFLAGS   *= $(shell "dpkg-buildflags --get CFLAGS; dpkg-buildflags --get CPPFLAGS")
| +QMAKE_CXXFLAGS *= $(shell "dpkg-buildflags --get CXXFLAGS; dpkg-buildflags --get CPPFLAGS")
| +QMAKE_LFLAGS   *= $(shell dpkg-buildflags --get LDFLAGS)
`----[...]

The rules file
https://salsa.debian.org/bacula-team/bacula/blob/master/debian/rules
just contains the usual:

,----
| # enable bindnow
| # https://wiki.debian.org/HardeningWalkthrough
| export DEB_BUILD_MAINT_OPTIONS = hardening=+all
|
| # avoid unnecessary linking
| # https://wiki.debian.org/HardeningWalkthrough
| export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed
`----

Maybe this gives you a hint at what to do and where.

S!

--
Sigmentation fault. Core dumped.

Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Otto Kekäläinen-4
In reply to this post by Otto Kekäläinen-4
So apparently the 'D_FORTIFY_SOURCE=2' is in CPPFLAGS (not read by
cmake) but not in CXXFLAGS (read by cmake)[1].

So maybe I should define?
CXXFLAGS=$(CXXFLAGS) $(CPPFLAGS)

This is the current state of mysqld, should I be happy with this or is
it relevant that all functions are protected?

hardening-check --verbose --color  mysqld
mysqld:
 Position Independent Executable: yes
 Stack protected: yes
 Fortify Source functions: yes (some protected functions found)
unprotected: strcpy
unprotected: strcat
unprotected: recv
unprotected: snprintf
unprotected: getcwd
unprotected: readlink
unprotected: memset
unprotected: poll
unprotected: fread
unprotected: fgets
unprotected: strncpy
unprotected: sprintf
unprotected: stpcpy
unprotected: strncat
unprotected: memcpy
unprotected: read
unprotected: confstr
unprotected: pread64
unprotected: memmove
unprotected: gethostname
protected: strcpy
protected: snprintf
protected: vfprintf
protected: memset
protected: poll
protected: vasprintf
protected: fread
protected: strncpy
protected: sprintf
protected: vsprintf
protected: memcpy
protected: fdelt
protected: realpath
protected: pread64
protected: vsnprintf
protected: fprintf
protected: memmove
protected: printf
 Read-only relocations: yes
 Immediate binding: yes



[1] https://cmake.org/Bug/view.php?id=12928

Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Niels Thykier
Otto Kekäläinen:
> So apparently the 'D_FORTIFY_SOURCE=2' is in CPPFLAGS (not read by
> cmake) but not in CXXFLAGS (read by cmake)[1].
>
> So maybe I should define?
> CXXFLAGS=$(CXXFLAGS) $(CPPFLAGS)
>

You have to with cmake, yes.  I believe debhelper carries a similar work
around (for CXXFLAGS + CFLAGS with cmake).

> This is the current state of mysqld, should I be happy with this or is
> it relevant that all functions are protected?
>
> hardening-check --verbose --color  mysqld
> [...]
>
>
> [1] https://cmake.org/Bug/view.php?id=12928
>

This is where blhc log check (where possible) is better as it will tell
you if all compiler invocations had the hardening flags.  If they do,
then you are good.
  It is "perfectly" normal for a correctly hardened binary to have
unhardened functions as the compiler *can* in some cases prove the
hardening is unnecessary.

On a related note, lintian has a list of symbols that it ignores for the
purpose of assuming/checking if a binary is hardened or not and it has
no hardened functions.  Mind you, that list is an accuracy trade-off (to
prefer false-negatives over false-positives due to the number of
false-positive complaints).

Thanks,
~Niels

Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Andrey Rahmatullin-3
In reply to this post by Otto Kekäläinen-4
On Fri, Apr 05, 2019 at 09:55:34PM +0300, Otto Kekäläinen wrote:
> Hello!
>
> > > Is there any hardening flag / cmake expert around who could help me
> > > get the hardening flags perfect in MariaDB 10.3?
> > Start with https://wiki.debian.org/Hardening#Notes_for_packages_using_CMake
>
> I've read this section many times over but I don't get it. A
> workaround is presented but since we are on a new debhelper it is
> advised not to be used.
The actual words are "only with compat=9 and dh_auto* commands!".
You are not using dh_auto_*.

> > > d/rules:
> > > https://salsa.debian.org/mariadb-team/mariadb-10.3/blob/master/debian/rules
> > One of the problems is using $(MAKE) instead of dh_auto_build and so on.
> > There are other problems in this file.
> Since the build command is constructed in the
> override_dh_auto_configure stanza this is the only way I am aware that
> I can pass it on to dh_auto_build.
Sorry, I don't understand this.

> I am happy to try out alternative ways if you have concrete suggestions
> on how to refactor the d/rules file
I'd suggest starting from scratch.

--
WBR, wRAR

signature.asc (911 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Andrey Rahmatullin-3
In reply to this post by Otto Kekäläinen-4
On Fri, Apr 05, 2019 at 10:17:16PM +0300, Otto Kekäläinen wrote:
> So apparently the 'D_FORTIFY_SOURCE=2' is in CPPFLAGS (not read by
> cmake) but not in CXXFLAGS (read by cmake)[1].
That's what the wiki page says, yes.

> So maybe I should define?
> CXXFLAGS=$(CXXFLAGS) $(CPPFLAGS)
That's the spirit of what the wiki page says. This exact code probably
won't work, just do what the wiki page says.

--
WBR, wRAR

signature.asc (911 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Andrey Rahmatullin-3
In reply to this post by Sven Hartge-5
On Fri, Apr 05, 2019 at 09:07:06PM +0200, Sven Hartge wrote:
> CMake is a bit "special" in that regard. To get the right hardening
> flags to work for some parts of Bacula, we had to include the following
> patch to kind-of brute force the flags:
> https://salsa.debian.org/bacula-team/bacula/blob/master/debian/patches/debian/enable-hardening-for-qmake
qmake != CMake.

--
WBR, wRAR

signature.asc (911 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Sven Hartge-5
Andrey Rahmatullin <[hidden email]> wrote:
> [-- text/plain, encoding quoted-printable, charset: utf-8, 11 lines --]

> On Fri, Apr 05, 2019 at 09:07:06PM +0200, Sven Hartge wrote:
>> CMake is a bit "special" in that regard. To get the right hardening
>> flags to work for some parts of Bacula, we had to include the following
>> patch to kind-of brute force the flags:
>> https://salsa.debian.org/bacula-team/bacula/blob/master/debian/patches/debian/enable-hardening-for-qmake

> qmake != CMake.

Yes, I noticed this the moment I hit sent. Must have been more tired
than I thought I was yesterday.

Grüße,
Sven

--
Sigmentation fault. Core dumped.

Reply | Threaded
Open this post in threaded view
|

Re: Seeking hardening flag / blhc expoert

Otto Kekäläinen-4
In reply to this post by Otto Kekäläinen-4
Hello!

Thanks everybody for the pointers. I fixed it now with:

Subject: [PATCH] Ensure cmake builds also apply CPPFLAGS flags for hardening
 to fully work

---
 debian/rules | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/debian/rules b/debian/rules
index 3a16f8bfa..2e7536b9c 100755
--- a/debian/rules
+++ b/debian/rules
@@ -7,6 +7,11 @@ export DH_VERBOSE=1
 export DEB_BUILD_MAINT_OPTIONS = hardening=+all
 DPKG_EXPORT_BUILDFLAGS = 1
 include /usr/share/dpkg/default.mk
+# CPPFLAGS are nor read by CMake, so copy them to CXXFLAGS
+# See why at https://cmake.org/Bug/view.php?id=12928
+# This is needed for e.g. all automatic Debian hardening flags to
apply on all cmake builds.
+CFLAGS+=$(CPPFLAGS)
+CXXFLAGS+=$(CPPFLAGS)

 # Only do a strict symbol checking on Linux
 ifneq (,$(filter linux,$(DEB_HOST_ARCH_OS)))

https://salsa.debian.org/mariadb-team/mariadb-10.3/commit/fc4f33cf40d0a10ef5d1992accd2af734ba96356

Results at:
https://salsa.debian.org/mariadb-team/mariadb-10.3/-/jobs/154355