Quantcast

State of emdebian / Future plans

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

State of emdebian / Future plans

Janek Filus
Hello,

Are some people still working on this project or is it dead? We would
like to know if there is still some work going on and what the future
plans are - looking through the archives seems that no one really
manages the homepage and its archives (full of spam).


Thanks for any answer,
Regards

Janek



--
bytes at work
Technoparkstrasse 7
CH-8406 Winterthur
Switzerland

phone: +41 52 213 79 79

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: State of emdebian / Future plans

wookey-4
On 2017-03-10 15:19 +0100, Janek Filus wrote:
> Hello,
>
> Are some people still working on this project or is it dead? We would
> like to know if there is still some work going on and what the future
> plans are - looking through the archives seems that no one really
> manages the homepage and its archives (full of spam).

Development of emdebian stopped 2-3? years ago. The repository
is still used to supply stable cross-toolchains, and I'll maintain
that for as long as jessie is supported.

Various outputs of the project are still used, like multistrap.

The web pages are kept up but not much effort goes into them.

Where is full of spam? I'm happy to tidy that up if I can.

The mailing list and IRC channel are still used from time to time.

So we are very much in maintenance/gentle-decline mode these days.

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: State of emdebian / Future plans

Baurzhan Ismagulov
Hello Janek,

On Fri, Mar 10, 2017 at 04:10:14PM +0000, Wookey wrote:
> > Are some people still working on this project or is it dead? We would
> > like to know if there is still some work going on and what the future
> > plans are - looking through the archives seems that no one really
> > manages the homepage and its archives (full of spam).

It depends pretty much on what you mean with "this project".

Wookey provides the toolchains. @Wookey: Does "as long as jessie is supported"
mean that stretch provides cross-compiler sources or binaries out of the box?

To my knowledge, SLIND [1] was the first Emdebian implementation. It provided
cross-build rules for a few packages and stripped some files (e.g., docs) and
dependencies (notably, perl ones) not necessary for embedded systems.

Neil has pushed the effort on making packages cross-buildable out of the box
[2, 3]. He released Emdebian distributions [4]. Grip provided stripped Debian
binary packages. Crush provided cross-built binary packages. The effort has
been abandoned due to lack of resources.

Meanwhile, SLIND's cross-building effort has also been abandoned for new
projects, due to the same reasons, in favor of native building. Also, a build
system has been added, and the project has been open-sourced as Isar [5]. It
relies on mainstream tools like BitBake, dpkg-buildpackage, multistrap and meta
layering similar to that of Yocto for easier maintenance of upstream code.

ELBE [6] is a tool for building root filesystems from apt repositories based on
an XML description.

Meta-debian / Deby cross-builds Debian sources using the Yocto infrastructure.
They contribute to OSS projects, including Debian [8].

In other words, companies continue using Debian in their products [9], it's too
valuable to re-invent it in-house. The efforts are somewhat distributed ATM,
although we are talking how we could improve things. A central policy and
co-ordination are currently missing, and I find that ok for the time being --
implementation and persistence first. So, if you would like to use Debian for
your projects -- that's a good choice, and there are tools for that; at the
same time, it's still a long way to a bigger and more organized community. If
we could help, please let us know.

Regarding the web pages -- I had the problem that there were several versions
of the same page (e.g., on building cross-compilers, or about multistrap), some
of which were outdated. @Wookey: Is this list an appropriate place to provide
suggestions on that?

References:

1. http://www.emdebian.org/News/2006/20060225.html
2. https://lists.debian.org/debian-devel/2007/11/msg00118.html
3. https://lists.debian.org/debian-devel/2008/02/msg00520.html
4. http://www.emdebian.org/emdebian/flavours.html
5. https://github.com/ilbers/isar
6. https://elbe-rfs.org/
7. https://github.com/meta-debian/meta-debian
8. http://elinux.org/images/e/ef/ContributingOSS_Yoshitake_Kobayashi-r3.pdf
9. http://elinux.org/images/0/09/ELC-2017-isar.pdf

With kind regards,
Baurzhan.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: State of emdebian / Future plans

Helmut Grohne
(Please CC me in replies.)

On Sun, Mar 12, 2017 at 07:00:28PM +0100, Baurzhan Ismagulov wrote:
> Wookey provides the toolchains. @Wookey: Does "as long as jessie is supported"
> mean that stretch provides cross-compiler sources or binaries out of the box?

Yes, stretch includes cross compilers maintained by gcc maintainer
Matthias Klose targeting a fair number of architectures.

> Neil has pushed the effort on making packages cross-buildable out of the box
> [2, 3]. He released Emdebian distributions [4]. Grip provided stripped Debian
> binary packages. Crush provided cross-built binary packages. The effort has
> been abandoned due to lack of resources.

With unstable now containing cross toolchains for a while now, our focus
has shifted. I have spent considerable effort at making packages in
unstable cross build out of the box. My current numbers are roughly:

 * 12000 source packages build architecture-dependent binary packages.
   (Others are not relevant to cross building.)
 * 6000 have satisfiable cross Build-Depends on at least one
   architecture.
 * I attempted cross building 2300 of those (selection mostly based on
   popcon) for random architectures.
 * 1300 cross built successfully. (Successful builds that failed using
   cross tools are already excluded here.)
 * I filed lots of bugs with patches. Currently, 192 distinct bugs
   affecting cross buildability of 630 source packages are recorded.
   A larger number (~400) of bugs has been fixed in stretch already.

Thus we can safely conclude that 10% of stretch will be cross buildable
out of the box.

> In other words, companies continue using Debian in their products [9], it's too
> valuable to re-invent it in-house. The efforts are somewhat distributed ATM,
> although we are talking how we could improve things. A central policy and
> co-ordination are currently missing, and I find that ok for the time being --
> implementation and persistence first. So, if you would like to use Debian for
> your projects -- that's a good choice, and there are tools for that; at the
> same time, it's still a long way to a bigger and more organized community. If
> we could help, please let us know.

Help would be welcome indeed. There are very many loose ends that
require effort to turn vanilla Debian into a reasonable
Yocto-competitor. Let me try to summarize a few that I have been working
on:
 * Generally making packages cross buildable. There are a number of very
   simple bug classes that simply require someone doing the work:
   + Autotools-based and cmake-based  packages that do not use
     dh_auto_configure often lack relevant --host flags (or their cmake
     equivalent).
   + .pc files shipped in /usr/lib/pkgconfig are not found during cross
     compilation and need to be moved to /usr/lib/<triplet>/pkgconfig.
   + Build-Dependencies on perl/python/ruby and other tools are often
     used as build tools rather than libraries. Thus many (but not all)
     need to be annotated with :any or :native.
   + A lot of packages call into plain pkg-config when they need to call
     the <hosttriplet>-pkg-config. Sometimes this happens due to using
     AC_CHECK_PROG instead of AC_CHECK_TOOL or because a Makefile
     hardcodes use of pkg-config.
   In general, the class of bugs that can be fixed in less than an hour
   is still very much populated. But there are also a number of harder
   issues.
 * Making Debian "bootstrappable", i.e. making it possible to cross
   compile a base system from source. This mostly requires adding build
   profiles in the relevant places and largely is a solved problem. For
   the embedded use case though, more profiles will be needed to
   optionally disable unnecessary features (i.e. what Emdebian Crush
   did). For instance, disabling selinux could noticeably reduce the
   build time. For many architectures, cross boostrapping mostly
   works[1] already.
 * Simplifying foreign installations. As far as I can tell, the current
   state of the art in installing packages still is multistrap. It
   allows foreign package installation albeit leaving them unconfigured.
   Configuration can be done using qemu-user-static or by rewriting the
   relevant scripts. I think neither approach scales and we should move
   to running standard maintainer scripts outside the chroot they are
   operating on. That feature goes by the name DPKG_ROOT[2] and
   currently, you can only experiment with it by passing
   --force-script-chrootless to dpkg. Adding support for DPKG_ROOT to
   maintainer scripts and figuring out its semantics is present work.
 * Cross compiling to from linux to non-linux or from glibc to non-glibc
   (e.g. musl) currently fails, because there are file conflicts in
   headers. We'll have to move[3] standard headers (e.g. limits.h) to
   /usr/include/<triplet>. The initial patches are there, but someone
   needs to perform an archive rebuild to see what breaks.

I am also interested in hearing more from cross compilation users to be
able to use a better prioritization mechanism than popcon. Anyone
interested in working on these topic is invited to get in touch with me.
The preferred ways are the #debian-bootstrap IRC channel on OFTC and
[hidden email]. I am not subscribed to [hidden email].

Helmut

[1] https://wiki.debian.org/HelmutGrohne/rebootstrap
[2] https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap
[3] https://wiki.debian.org/Multiarch/GlibcHeaders

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: State of emdebian / Future plans

wookey-4
On 2017-03-13 12:40 +0100, Helmut Grohne wrote:

Thanks, Helmut, for your comprehensive summary of current status.

As you can see, Helmut is currently the one doing most of the running
on this, and we are making steady (but slow) progress on making stock
debian as small and cross-buildable as emdebian was.

It was always the plan to merge what we discovered in the Emdebian
effort back into Debian proper, and that work has gone quite well
(cross-toolchains, cross-buildability, build-profiles, multiarch
co-installability). But there are still hundreds of things to fix...

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Emdebian web page (was: State of emdebian / Future plans)

W. Martin Borgert
On 2017-03-13 13:10, Wookey wrote:
> As you can see, Helmut is currently the one doing most of the running
> on this, and we are making steady (but slow) progress on making stock
> debian as small and cross-buildable as emdebian was.

Maybe the emdebian web page should reflect those efforts and
changes in focus? The name "emdebian" is too nice to let it rot!

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Emdebian web page

Janek Filus

On 03/16/2017 09:53 PM, W. Martin Borgert wrote:
> On 2017-03-13 13:10, Wookey wrote:
>> As you can see, Helmut is currently the one doing most of the running
>> on this, and we are making steady (but slow) progress on making stock
>> debian as small and cross-buildable as emdebian was.
> Maybe the emdebian web page should reflect those efforts and
> changes in focus? The name "emdebian" is too nice to let it rot!
I agree, why not reflecting the efforts made and the open points of the
project on the webpage? It's not clear from my point of view - thats why
this discussion has started. It would be great having newer information
available about the project's current state, packages, toolchain issues
and meta-yocto integration.

--
bytes at work
Technoparkstrasse 7
CH-8406 Winterthur
Switzerland

phone: +41 52 213 79 79

Loading...