You are not logged in.

#1 2013-04-15 02:44:10

sezaru
New member
Registered: 2013-04-15
Posts: 2

[Solved] ZFS memory leak

Ok, the problem was not actually a memory leak, but simply a feature, ZFS will use all the available RAM to cache files, maybe this is obvious for the others, but for people new to this fs this was kinda confunsing, now I added the arcmax=536870912 to limite the cache to just 512Mb and everything is working great.

I think some note about it is needed in the wiki page of the instalation of ZFS to help others who dont know about this feature.

Edit:

Ok guys, I changed the title and are editing the top of the message because I found what is actually happening.

ZFS is leaking every bit of anything it reads or writes to the system, I found out getting some big file to the HD and then doing:

$ cat big_file > /dev/null

As the file is read from the Hd the RAM memory will grow with it, so it's clear that the culprit is zfs.

I've tried to install the stable gentoo-sources version 3.7.10 but the result was the same.

There it is the zfs and spl versions I have instaled:
zfs 0.6.1
zfs-kmod 0.6.1
spl 0.6.1

Original post:

Hello guys, I've got a new notebook (Dell Inspiron 7720) and instaled Funtoo in it with ZFS as the filesystem, this new notebook is somewhat similar in hardware to another notebook I have (Asus G74SX) , (the most noticable difference is that this new one have NVIDIA Optimus and the other one have just a NVIDIA card) so I've just copied almost all the same configuration from the funtoo in the other notebook to this new one, and just made some modifications in the kernel to support the ZFS and the intel card.

Well, after it finished the instalation, it is working great, but it looks like the system is somewhat not releasing any memory from the RAM, when I fresh boot, it starts consuming more than 800Mb, the other system consumes just 150Mb.

And as I continue to use the system, more and more RAM is being consumed, and when it reaches the maximum of my RAM (8Gb) it will start doing swap.

The most noticable change I can think between the 2 kernel configurantion is the change from CFQ schedule to no-op as the ZFS guide recomends, the other obvious change is the intel graphics driver module.

I normally like to put more information about the problem here, but I'm not sure what to put here as I don't kknow what is causing it, so ask me whatever you guys think is necessary to find the problem.

make.conf:

MARCH="native"
COREFLAGS="-O2 -pipe -ggdb"
CFLAGS="-march=${MARCH} ${COREFLAGS}"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j9 -l8"
CHOST="x86_64-pc-linux-gnu"
ACCEPT_KEYWORDS="~amd64"
ABI_X86="64 32"

PORTAGE_NICENESS="15"
FEATURES="parallel-fetch userfetch distlocks mini-manifest splitdebug"
EMERGE_DEFAULT_OPTS="--ask --verbose --keep-going=y --quiet-build=n --with-bdeps=y --jobs=8 --load-average=8"

LOCAL_OVERLAY="/home/kiwi/configs/overlay"
PORTDIR_OVERLAY="${LOCAL_OVERLAY}"
source /var/lib/layman/make.conf

INPUT_DEVICES="evdev synaptics keyboard mouse"
VIDEO_CARDS="intel nvidia"
ALSA_CARDS="hda-intel"
ALSA_PCM_PLUGINS="*"
LINGUAS="en en_US pt_BR ko ko_KR ja ja_JP zh zh_CN zh_TW"

USE_CPU="acpi mmx sse sse2 sse3 ssse3 sse4 sse4_1 sse4_2 avx dts"
USE_SYS="aspell c++0x fontconfig glibc-omitfp minimal opencl opengl symlink threads"
USE_MEDIA="aac bluray ffmpeg svg theora v4l vdpau x264 xv"
USE_NO="-gpm -perl -oss"
USE="${USE_CPU} ${USE_SYS} ${USE_MEDIA} ${USE_NO}"

Kernel:

Linux localhost 3.8.7-gentoo #1 SMP PREEMPT Sun Apr 14 00:46:08 UTC 2013 x86_64 Intel(R) Core(TM) i7-3630QM CPU @ 2.40GHz GenuineIntel GNU/Linux

lspci:

00:00.0 Host bridge: Intel Corporation 3rd Gen Core processor DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09)
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)
00:16.0 Communication controller: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1b.0 Audio device: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4)
00:1c.4 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 5 (rev c4)
00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation HM77 Express Chipset LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 7 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)
00:1f.3 SMBus: Intel Corporation 7 Series/C210 Series Chipset Family SMBus Controller (rev 04)
01:00.0 VGA compatible controller: NVIDIA Corporation GK107 [GeForce GT 650M] (rev ff)
02:00.0 Network controller: Intel Corporation Centrino Wireless-N 2230 (rev c4)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8101E/RTL8102E PCI Express Fast Ethernet controller (rev 05)

Thanks!

Edit: Just to give some more information, I was copying a 30Gb file from one computer to this one via http and the RAM started to increase as the file was being copied, It looked like the file was being copied to the RAM instead of the harddrive, when the file reached 5.7Gb it stopped increasing the RAM. free shows that the system is using 6.2Gb of RAM, and the only thing I have opened is conkeror with the buffer to this forum...

             total       used       free     shared    buffers     cached
Mem:          7847       6301       1546          0          0        251
-/+ buffers/cache:       6049       1797
Swap:         9215         39       9176

Last edited by sezaru (2013-04-15 05:52:24)

Offline

#2 2013-04-17 14:40:05

404_Error
Core Team Member
From: Quebec city, Quebec, Canada
Registered: 2010-10-08
Posts: 511

Re: [Solved] ZFS memory leak

ZFS (on Solaris) is known to « eat » large amounts of system memory for its own buffer I would not be suprised it this is also happens on Linux...

Offline

Board footer

Powered by FluxBB