kernel CAN'T EMERGE -- "Too many open files" HELP!

patty.yivi · May 18, 2016

Please review and help if you can.

Below there is reports of failure for trying to emerge debian, vanilla and gentoo sources.

As you can see the core issue is caused by:

"Too many open files"

How to control allowed number of open files?

For debian-sources:

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] sys-kernel/debian-sources-4.5.2:4.5.2::gentoo  USE="-binary" 0 KiB

Total: 1 package (1 new), Size of downloads: 0 KiB


>>> Verifying ebuild manifests
>>> Running pre-merge checks for sys-kernel/debian-sources-4.5.2

>>> Emerging (1 of 1) sys-kernel/debian-sources-4.5.2::gentoo
 * linux_4.5.2.orig.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                [ ok ]
 * linux_4.5.2-1.debian.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                            [ ok ]
>>> Unpacking source...
>>> Unpacking linux_4.5.2.orig.tar.xz to /mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/work
>>> Unpacking linux_4.5.2-1.debian.tar.xz to /mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/work
>>> Source unpacked in /mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/work
>>> Preparing source in /mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/work/linux-4.5.2 ...
 * Applying version.patch ...                                                                                                                                                  [ ok ]
 * Applying uname-version-timestamp.patch ...                                                                                                                                  [ ok ]
 * Applying kernelvariables.patch ...                                                                                                                                          [ ok ]
 * Applying gitignore.patch ...                                                                                                                                                [ ok ]
 * Applying mips-disable-werror.patch ...                                                                                                                                      [ ok ]
 * Applying arch-sh4-fix-uimage-build.patch ...                                                                                                                                [ ok ]
 * Applying powerpcspe-omit-uimage.patch ...                                                                                                                                   [ ok ]
 * Applying Kbuild-kconfig-Verbose-version-of-listnewconfig.patch ...                                                                                                          [ ok ]
 * Applying modpost-symbol-prefix.patch ...                                                                                                                                    [ ok ]
 * Applying tools-perf-version.patch ...                                                                                                                                       [ ok ]
 * Applying tools-perf-install.patch ...                                                                                                                                       [ ok ]
 * Applying drivers-media-dvb-usb-af9005-request_firmware.patch ...                                                                                                            [ ok ]
 * Applying iwlwifi-do-not-request-unreleased-firmware.patch ...                                                                                                               [ ok ]
 * Applying firmware_class-return-specific-errors-from-file-read.patch ...                                                                                                     [ ok ]
 * Applying firmware_class-log-every-success-and-failure.patch ...                                                                                                             [ ok ]
 * Applying firmware-remove-redundant-log-messages-from-drivers.patch ...

 * Failed Patch: firmware-remove-redundant-log-messages-from-drivers.patch !
 *  ( /mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/work/debian/patches/bugfix/all/firmware-remove-redundant-log-messages-from-drivers.patch )
 *
 * Include in your bugreport the contents of:
 *
 *   /mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/temp/firmware-remove-redundant-log-messages-from-drivers.patch.out

 * ERROR: sys-kernel/debian-sources-4.5.2::gentoo failed (prepare phase):
 *   Failed Patch: firmware-remove-redundant-log-messages-from-drivers.patch!
 *
 * Call stack:
 *     ebuild.sh, line  133:  Called src_prepare
 *   environment, line 2280:  Called epatch '-p1' '/mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/work/debian/patches/bugfix/all/firmware-remove-redundant-log-messages-from-drivers.patch'
 *   environment, line  882:  Called die
 * The specific snippet of code:
 *               die "Failed Patch: ${patchname}!";
 *
 * If you need support, post the output of `emerge --info '=sys-kernel/debian-sources-4.5.2::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-kernel/debian-sources-4.5.2::gentoo'`.
 * The complete build log is located at '/mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/temp/build.log'.
 * The ebuild environment file is located at '/mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/temp/environment'.
 * Working directory: '/mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/work/linux-4.5.2'
 * S: '/mnt/_var_tmp/_var/portage/sys-kernel/debian-sources-4.5.2/work/linux-4.5.2'

However also similar problem comes for e.g. vanilla-sources

y y # emerge -va vanilla-sources

 * IMPORTANT: 29 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.


 * IMPORTANT: config file '/etc/portage/package.use' needs updating.
 * See the CONFIGURATION FILES section of the emerge
 * man page to learn how to update config files.

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ~] sys-kernel/vanilla-sources-4.5.4:4.5.4::gentoo  USE="-build -symlink" 0 KiB

Total: 1 package (1 new), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] Y

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) sys-kernel/vanilla-sources-4.5.4::gentoo
 * patch-4.5.4.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                         [ ok ]
 * linux-4.5.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                       [ ok ]
>>> Preparing to unpack ...
>>> Unpacking source...
>>> Unpacking linux-4.5.tar.xz to /mnt/_var_tmp/_var/portage/sys-kernel/vanilla-sources-4.5.4/work
 * Excluding Patch #5000_enable-additional-cpu-optimizations-for-gcc.patch ...                                                                                                 [ ok ]
 * Excluding Patch #5015_kdbus*.patch ...                                                                                                                                      [ ok ]
patch: **** Can't create temporary file /mnt/_var_tmp/_var/portage/sys-kernel/vanilla-sources-4.5.4/temp/poovxW5x : Too many open files
 * Failed to dry-run patch patch-4.5.4.patch
 * Please attach /mnt/_var_tmp/_var/portage/sys-kernel/vanilla-sources-4.5.4/temp/patch-4.5.4.err to any bug you may post.
 * ERROR: sys-kernel/vanilla-sources-4.5.4::gentoo failed (unpack phase):
 *   Unable to dry-run patch on any patch depth lower than 5.
 *
 * Call stack:
 *     ebuild.sh, line  133:  Called src_unpack
 *   environment, line 3597:  Called kernel-2_src_unpack
 *   environment, line 1960:  Called unipatch '/mnt/_var_tmp/_var/portage/sys-kernel/vanilla-sources-4.5.4/distdir/patch-4.5.4.xz  '
 *   environment, line 4339:  Called die
 * The specific snippet of code:
 *                   die "Unable to dry-run patch on any patch depth lower than 5.";
 *
 * If you need support, post the output of `emerge --info '=sys-kernel/vanilla-sources-4.5.4::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-kernel/vanilla-sources-4.5.4::gentoo'`.
 * The complete build log is located at '/mnt/_var_tmp/_var/portage/sys-kernel/vanilla-sources-4.5.4/temp/build.log'.
 * The ebuild environment file is located at '/mnt/_var_tmp/_var/portage/sys-kernel/vanilla-sources-4.5.4/temp/environment'.
 * Working directory: '/mnt/_var_tmp/_var/portage/sys-kernel/vanilla-sources-4.5.4/work/linux-4.5.4'
 * S: '/mnt/_var_tmp/_var/portage/sys-kernel/vanilla-sources-4.5.4/work/linux-4.5.4'

and also as well for gentoo-sources:

y y # emerge -v gentoo-sources
...
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  NS    ] sys-kernel/gentoo-sources-4.4.6:4.4.6::gentoo [4.1.12:4.1.12::gentoo] USE="-build -experimental -kdbus -symlink" 0 KiB

Total: 1 package (1 in new slot), Size of downloads: 0 KiB

!!! The following installed packages are masked:
- net-dialup/ppp-2.4.5-r1::gentoo (masked by: package.mask)
/mnt/_tinies/_Funtoo/portage/profiles/package.mask/00-gentoo:
# Sergey Popov <pinkbyte@gentoo.org> (28 Aug 2014)
# Security mask, wrt bug #519650
# If your application is broken due to this mask,
# please file a separate bug report

- net-nds/openldap-2.4.30::gentoo (masked by: package.mask)
/mnt/_tinies/_Funtoo/portage/profiles/package.mask/00-gentoo:
# Sergey Popov <pinkbyte@gentoo.org> (20 Mar 2014)
# Security mask of vulnerable versions, wrt bug #424167

- dev-vcs/subversion-1.7.14::gentoo (masked by: package.mask)
/mnt/_tinies/_Funtoo/portage/profiles/package.mask/00-gentoo:
# Sergey Popov <pinkbyte@gentoo.org> (26 Apr 2016)
# Security mask of older branches, wrt bug #5567810

- x11-base/xorg-server-1.17.4::gentoo (masked by: package.mask)
/mnt/_tinies/_Funtoo/portage/profiles/package.mask/funtoo-staging:
# FL-2232: Not fully-baked version of xorg-server. Masking.

For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.



>>> Verifying ebuild manifests

>>> Emerging (1 of 1) sys-kernel/gentoo-sources-4.4.6::gentoo
 * linux-4.4.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                       [ ok ]
 * genpatches-4.4-8.base.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                           [ ok ]
 * genpatches-4.4-8.extras.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                         [ ok ]
>>> Preparing to unpack ...
>>> Unpacking source...
>>> Unpacking linux-4.4.tar.xz to /mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/work
>>> Unpacking genpatches-4.4-8.base.tar.xz to /mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/work/patches
>>> Unpacking genpatches-4.4-8.extras.tar.xz to /mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/work/patches
 * Excluding Patch #5000_enable-additional-cpu-optimizations-for-gcc.patch ...                                                                                                 [ ok ]
 * Excluding Patch #5015_kdbus*.patch ...                                                                                                                                      [ ok ]
 * Applying 1000_linux-4.4.1.patch (-p1) ...
patch: **** can't open file sound/soc/codecs/wm5110.c : Too many open files                                                                                                    [ !! ]
 * Failed to apply patch 1000_linux-4.4.1.patch
 * Please attach /mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/temp/1000_linux-4.4.1.err to any bug you may post.
 * ERROR: sys-kernel/gentoo-sources-4.4.6::gentoo failed (unpack phase):
 *   Failed to apply 1000_linux-4.4.1.patch on patch depth 1.
 *
 * Call stack:
 *     ebuild.sh, line  133:  Called src_unpack
 *   environment, line 3602:  Called kernel-2_src_unpack
 *   environment, line 1963:  Called unipatch '  /mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/distdir/genpatches-4.4-8.base.tar.xz /mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/distdir/genpatches-4.4-8.extras.tar.xz '
 *   environment, line 4334:  Called die
 * The specific snippet of code:
 *                           die "Failed to apply ${i/*\//} on patch depth ${PATCH_DEPTH}.";
 *
 * If you need support, post the output of `emerge --info '=sys-kernel/gentoo-sources-4.4.6::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-kernel/gentoo-sources-4.4.6::gentoo'`.
 * The complete build log is located at '/mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/temp/build.log'.
 * The ebuild environment file is located at '/mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/temp/environment'.
 * Working directory: '/mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/work/linux-4.4.6-gentoo'
 * S: '/mnt/_var_tmp/_var/portage/sys-kernel/gentoo-sources-4.4.6/work/linux-4.4.6-gentoo'

jmmc · May 18, 2016

Q: Did you achieve first boot with the default debian-sources - or are you just trying to get to first (initial) boot?

Q2: Do you have hardware or a filesystem not supported by the stock debian-sources kernel?

patty.yivi · May 18, 2016

So far I've booted with the previous kernel, compiled yet within gentoo system, namely: gentoo-sources-4.4.0-r1

I've normal laptop, Lenovo Y550, with nvidia and intel cpu, and filesystems on my computer are: xfs (root '/'), reiserfs (portage, ccache), ext2 (tmp, /var/tmp), ntfs (miscellaneous data) and ntfs is mounted as ntfs-3g

Besides, also no other kernel brand wants to EMERGE (install), including gentoo-sources.

Gentoo kernel so far has worked, emerged and compiled well, but after switching to Funtoo can't emerge at all.

Also to mention /var/tmp free space is about 19GB, and maybe more when renewed attempt of installing (emerge -v debian-sources) pures what left from the previous one.

Oleg Vinichenko · May 20, 2016

i'm not entirely sure but maybe you hitting ulimit problems. Do you have ulimit specified anywhere? ulimit -n shows it.

But i guess dirty workaround could be:

rm -rf /var/tmp/portage

patty.yivi · May 21, 2016

ulimit -n

yields:

1024

How to handle ulimit at all and what to do next with this value, please?

Sent from my Nexus 7 using Tapatalk

Oleg Vinichenko · May 21, 2016

this value is much lower than default, which is 16000 for open files. Normally this defined in /etc/security/limits.conf. There're several ways to change the values but first of all try to find out why you got unusual value. Did you change that?

patty.yivi · May 21, 2016

As might already presume this is the first time I hear about it at all ;)

What's more there isn't anything like /etc/security folder at all on my system.

Wait, has it anything to do with pam? I entirely resigned of emerging pam and set use flags for it for ebuilds not to incorporate pam dependencies...

patty.yivi · May 21, 2016

After doing some research about ulimit and linux limits at all through Google etc. have tried set such two configuration files:

/etc/security/limits.conf

*              hard    nofile          1028096
*              soft    nofile          1024000

and

/etc/limits

* N1024000
sysadm  -
root -
portage -

however, after logging out in in it has taken exactly no effect:

# ulimit -n
1024

In order to make possible emerging debian-sources, have done simple thing: ulimit -n 1024000
and it works:

# ulimit -n
1024000

of course just within the login session, and then it flees.
It helped to emerge debian-sources, but the problem of setting the limits values which would survive system reboot is still intact.

patty.yivi · May 22, 2016

------ EDIT:

It turns out settings abovementioned work! :)

Those in: /etc/limits

while these in: /etc/security/limits.conf bring absolutely no effect!

However only after rebooting computer they've came into effect, not just after logging out and in as people write in tutorials...

And they hold just for graphical terminal, for ordinary user or when made su, for the text console when logging in as root it still says:

# ulimit -n

1024

[\code]

but ordinary user's console respects these settings.

Oleg Vinichenko · May 22, 2016

good. but what was the original problem? did you manually change the limits before? default values (not like yours) certainly enough for merging debian-sources. what i'm trying to figure whether it's manual change or not.

patty.yivi · May 22, 2016

As you probably read already about the original problem is covering from Gentoo.

As you can see step by step it succeeds iteratively solving such issues like the abovementioned. Previously as yet branded as Gentoo the system worked well without any limits settings. Conversion something introduced.

Sent from my Nexus 7 using Tapatalk

cardinal · May 22, 2016

Funtoo's sys-libs/pam-1.1.8-r2.ebuild changes limits.conf:

# setting default number of open files to 16000, with the ability to
# push the limit up to 64000. This provides reasonable defaults for modern
# systems that need to handle things like slowloris in defaultconfigs.

echo "* soft nofile 16000" >> ${D}/etc/security/limits.conf || die "limits set fail"
echo "* hard nofile 64000" >> ${D}/etc/security/limits.conf || die "limits set fail"

Funtoo current x86_64
rj@funtoo ~ $ equery b /etc/security/limits.conf
* Searching for /etc/security/limits.conf ...
sys-libs/pam-1.1.8-r2 (/etc/security/limits.conf)
rj@funtoo ~ $ ulimit -n
16000

Gentoo stable x86_64
rj@gentoo ~ $ equery b /etc/security/limits.conf
* Searching for /etc/security/limits.conf ...
sys-libs/pam-1.2.1 (/etc/security/limits.conf)
rj@gentoo ~ $ ulimit -n
1024

Oleg Vinichenko · May 23, 2016

ok, so this is yet again using existing install. please. install Funtoo from fresh stage. Converting existing Gentoo installation into Funtoo is unsupported (though, theoretically possible). You are hitting various discrepancies due to that, and this is not really Funtoo bug.

patty.yivi · May 23, 2016

It looks like only this one is yet on the battlefield, actually, and the fresh install with compiling everything anew seems too cumbersome, not worth it.

Sent from my Nexus 7 using Tapatalk

patty.yivi · June 1, 2016

ulimit -n 1024 is set 1024 on Gentoo as default value, probably such a security measure.

However interesting why Gentoo can emerge kernel with this and Funtoo can not?

kernel CAN'T EMERGE -- "Too many open files" HELP!

Question

Link to comment

Share on other sites

14 answers to this question

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites