Remove package file from cache as soon as it is installed

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Remove package file from cache as soon as it is installed

André Rodier
Hello,

Is there a way, when using apt to install packages, to delete the
package file from /var/cache as soon as it is installed?

I am running a package installation inside docker:stable, and it fails
in the middle, with no space left on device.

Otherwise, is there an option to increase the docker image before
installing it?

Thanks,
André

Reply | Threaded
Open this post in threaded view
|

Re: Remove package file from cache as soon as it is installed

Nemeth Gyorgy
2019. 09. 22. 17:29 keltezéssel, André Rodier írta:
> Is there a way, when using apt to install packages, to delete the
> package file from /var/cache as soon as it is installed?

apt-get clean

Reply | Threaded
Open this post in threaded view
|

Re: Remove package file from cache as soon as it is installed

André Rodier
On Sun, 2019-09-22 at 18:01 +0200, Nemeth Gyorgy wrote:
> 2019. 09. 22. 17:29 keltezéssel, André Rodier írta:
> > Is there a way, when using apt to install packages, to delete the
> > package file from /var/cache as soon as it is installed?
>
> apt-get clean
>
Thanks, I knew the command, so perhaps I did not explain properly.

Let's take an example:

> Step 7/24 : RUN apt -qq install -t buster-backports -y simple-cdd
> debian-archive-keyring  
>  ---> Running in
> e3e9a1948203                                                        
>      
>                                                                      
>                      
> WARNING: apt does not have a stable CLI interface. Use with caution
> in scripts.            
>                                                                      
>                      
> debian-archive-keyring is already the newest version
> (2019.1).                            
> debian-archive-keyring set to manually
> installed.                                          
> The following additional packages will be
> installed:                                      
>   bc binutils binutils-common binutils-x86-64-linux-gnu build-
> essential bzip2              
>   ca-certificates cpp cpp-8 curl dbus dctrl-tools debian-cd
> dirmngr                        
>   distro-info-data dose-distcheck dosfstools dpkg-dev fakeroot
> file                        
>   fontconfig-config fonts-dejavu-core fonts-droid-fallback fonts-
> noto-mono g++            
>   g++-8 gcc gcc-8 ghostscript gnupg gnupg-l10n gnupg-utils gpg gpg-
> agent                  
>   gpg-wks-client gpg-wks-server gpgconf gpgsm gpgv gsfonts hfsutils
> iso-codes              
>   isolinux krb5-locales libalgorithm-diff-perl libalgorithm-diff-xs-
> perl                  
>   libalgorithm-merge-perl libapparmor1 libapt-inst2.0 libarchive13
> libasan5                
>   libassuan0 libatomic1 libavahi-client3 libavahi-common-data
> libavahi-common3            
>   libbinutils libbsd0 libburn4 libc-dev-bin libc6-dev libcc1-0
> libcups2                    
>   libcupsfilters1 libcupsimage2 libcurl4 libdbus-1-3 libdpkg-perl
> libexpat1                
>   libfakeroot libfile-fcntllock-perl libfontconfig1 libfreetype6
> libgcc-8-dev              
>   libgdbm-compat4 libgdbm6 libgomp1 libgpgme11 libgpm2 libgs9 libgs9-
> common                
>   libgssapi-krb5-2 libicu63 libidn11 libijs-0.35 libisl19
> libisoburn1                      
>   libisofs6 libitm1 libjbig0 libjbig2dec0 libjpeg62-turbo libjte1
> libk5crypto3            
>   libkeyutils1 libkrb5-3 libkrb5support0 libksba8 liblcms2-2 libldap-
> 2.4-2                
>   libldap-common liblocale-gettext-perl liblsan0 liblua5.2-0
> libmagic-mgc                  
>   libmagic1 libmpc3 libmpdec2 libmpfr6 libmpx2 libncurses6
> libnetpbm10                    
>   libnghttp2-14 libnpth0 libnspr4 libnss3 libopenjp2-7 libpaper-
> utils                      
>   libpaper1 libpcre2-8-0 libperl5.28 libpng16-16 libpopt0
> libpsl5                          
>   libpython3-stdlib libpython3.7-minimal libpython3.7-stdlib
> libquadmath0                  
>   libreadline7 librpm8 librpmio8 librtmp1 libsasl2-2 libsasl2-modules
>   libsasl2-modules-db libsqlite3-0 libssh2-1 libssl1.1 libstdc++-8-
> dev
>   libtiff5 libtsan0 libubsan1 libwebp6 libxml2 libyaml-0-2 linux-
> libc-dev
>   lsb-base lsb-release lynx lynx-common make manpages manpages-dev
>   mime-support mtools netbase netpbm openssl patch perl perl-modules-
> 5.28
>   pinentry-curses poppler-data publicsuffix python-apt-common python3
>   python3-apt python3-chardet python3-debian python3-minimal
>   python3-pkg-resources python3-simple-cdd python3-six python3-yaml
> python3.7
>   python3.7-minimal readline-common reprepro rpm-common rsync
> sensible-utils
>   syslinux-common syslinux-utils tofrodos ucf wget xorriso xz-utils
> [...]
> Selecting previously unselected package g++-
> 8.                                                          
> Preparing to unpack .../044-g++-8_8.3.0-6_amd64.deb
> ...                                                
> Unpacking g++-8 (8.3.0-6)
> ...                                                                  
>          
> dpkg: error processing archive /tmp/apt-dpkg-install-x4yofK/044-g++-
> 8_8.3.0-6_amd64.deb (--unpack):    
>  error creating directory './usr/share/doc/gcc-8-base/C++': No space
> left on device                    
> tar: ./prerm: Cannot open: No space left on
> device                                                      
> tar: Exiting with failure status due to previous
> errors                                                
> dpkg-deb: error: tar subprocess returned error exit status
> 2                                            
> dpkg: error processing archive /tmp/apt-dpkg-install-x4yofK/045-
> g++_4%3a8.3.0-1_amd64.deb (--unpack):  
>  dpkg-deb --control subprocess returned error exit status
> 2                                            
> Selecting previously unselected package
> make.                                                          
> Preparing to unpack .../046-make_4.2.1-1.2_amd64.deb
> ...                                                
> Unpacking make (4.2.1-1.2)
> ...                                                                  
>        
> dpkg: error processing archive /tmp/apt-dpkg-install-x4yofK/046-
> make_4.2.1-1.2_amd64.deb (--unpack):    
>  unable to create '/usr/bin/make.dpkg-new' (while processing
> './usr/bin/make'): No space left on device
> dpkg-deb: error: paste subprocess was killed by signal (Broken
> pipe)                                    
> dpkg: unrecoverable fatal error,
> aborting:                                                            
>  
>  unable to create '/var/lib/dpkg/updates/tmp.i': No space left on
> device                                
> E: Sub-process /usr/bin/dpkg returned an error code
> (2)                                                
>

When I run this command, I would like the deb file to be removed as
soon as the package is installed, one by one.

No option for this, I suppose?

André

Reply | Threaded
Open this post in threaded view
|

Re: Remove package file from cache as soon as it is installed

Reco
In reply to this post by André Rodier
        Hi.

On Sun, Sep 22, 2019 at 04:29:54PM +0100, André Rodier wrote:
> Hello,
>
> Is there a way, when using apt to install packages, to delete the
> package file from /var/cache as soon as it is installed?

/usr/share/doc/apt/NEWS.Debian.gz:

apt (1.2~exp1) experimental; urgency=medium

  [ Automatic removal of debs after install ]
  After packages are successfully installed by apt(8),
  the corresponding .deb package files will be
  removed from the /var/cache/apt/archives cache directory.

  This can be changed by setting the apt configuration option
    "Binary::apt::APT::Keep-Downloaded-Packages" to "true". E.g:

  # echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' \
      > /etc/apt/apt.conf.d/01keep-debs

  Please note that the behavior of apt-get is unchanged. The
  downloaded debs will be kept in the cache directory after they
  are installed. To enable the behavior for other tools, you can set
  "APT::Keep-Downloaded-Packages" to false.


In short, either you use "apt <do> <foo>", or "apt-get -o
Binary::apt::APT::Keep-Downloaded-Packages=false <do> <foo>".

Reco

Reply | Threaded
Open this post in threaded view
|

Re: Remove package file from cache as soon as it is installed

David Wright-3
In reply to this post by André Rodier
On Sun 22 Sep 2019 at 16:29:54 (+0100), André Rodier wrote:
> Hello,
>
> Is there a way, when using apt to install packages, to delete the
> package file from /var/cache as soon as it is installed?
>
> I am running a package installation inside docker:stable, and it fails
> in the middle, with no space left on device.

I think you need to break up the apt command into several of them,
with clean in between (if needed: apt might clean automatically;
IDK as I use apt-get).

Install the dependencies of dependencies first, then the dependencies,
then the packages *with* those dependencies, ie starting from the
bottom of the tree of dependencies.

> Otherwise, is there an option to increase the docker image before
> installing it?

Cheers,
David.

Reply | Threaded
Open this post in threaded view
|

Re: Remove package file from cache as soon as it is installed

André Rodier
On Sun, 2019-09-22 at 11:08 -0500, David Wright wrote:

> On Sun 22 Sep 2019 at 16:29:54 (+0100), André Rodier wrote:
> > Hello,
> >
> > Is there a way, when using apt to install packages, to delete the
> > package file from /var/cache as soon as it is installed?
> >
> > I am running a package installation inside docker:stable, and it
> > fails
> > in the middle, with no space left on device.
>
> I think you need to break up the apt command into several of them,
> with clean in between (if needed: apt might clean automatically;
> IDK as I use apt-get).
>
> Install the dependencies of dependencies first, then the
> dependencies,
> then the packages *with* those dependencies, ie starting from the
> bottom of the tree of dependencies.
>
> > Otherwise, is there an option to increase the docker image before
> > installing it?
>
> Cheers,
> David.
>
Thanks David et al,

The issue was the number of inodes in my /var/lib/docker partition was
ridiculously small, I had to reformat the partition to increase the
number of inodes, and it is now working.

Th partition is 20G, with the big files option, perhaps it was the
reason of the small number of inodes?

And yes, I was using overlay2.

Thanks again for your help,

André



Reply | Threaded
Open this post in threaded view
|

Re: Remove package file from cache as soon as it is installed

Sven Hartge-5
André Rodier <[hidden email]> wrote:

> Th partition is 20G, with the big files option, perhaps it was the
> reason of the small number of inodes?

Yes, of course. Why did you chose this option when mkfs'ing the
filesystem?

Besides: Those special options are more or less a thing of the past,
where you really had to think about the future use of a filesystem,
whether you put many many small files needing many inodes (Squid Cache
Spool, INN2 Usenet Spool, qmail Maildir) or very big ones, who would get
overly fragmented, lowering the throughput when reading them, on it.

Today the default is fine for (nearly) every use-case. (And you know if
you have one of the special ones.)

Grüße,
Sven.

--
Sigmentation fault. Core dumped.

Reply | Threaded
Open this post in threaded view
|

Re: Remove package file from cache as soon as it is installed

André Rodier
On Sun, 2019-09-22 at 20:17 +0200, Sven Hartge wrote:

> André Rodier <[hidden email]> wrote:
>
> > Th partition is 20G, with the big files option, perhaps it was the
> > reason of the small number of inodes?
>
> Yes, of course. Why did you chose this option when mkfs'ing the
> filesystem?
>
> Besides: Those special options are more or less a thing of the past,
> where you really had to think about the future use of a filesystem,
> whether you put many many small files needing many inodes (Squid
> Cache
> Spool, INN2 Usenet Spool, qmail Maildir) or very big ones, who would
> get
> overly fragmented, lowering the throughput when reading them, on it.
>
> Today the default is fine for (nearly) every use-case. (And you know
> if
> you have one of the special ones.)
>
> Grüße,
> Sven.
>
Thanks, that make sense. Thanks a lot.

André

Reply | Threaded
Open this post in threaded view
|

Re: Remove package file from cache as soon as it is installed

André Rodier
In reply to this post by Sven Hartge-5
Thanks again,

So, I now have my little project working again.

It build an ISO image from a simple YAML file, in three flavours:

- LUKS
- LVM
- RAID

For those interested, here the link:

https://github.com/progmaticltd/debian-iso-builder/tree/buster

I am sure this can be useful.

Enjoy!

--
André