eix behaives strage

erikr · February 19, 2016

Hi,

Recently I experience problems with eix similar to what is described in http://forums.funtoo.org/topic/177-lots-of-xfce-removed-from-portage/?p=831.

I found the problem with qt-creator;

# eix qt-creator
[?] dev-qt/qt-creator
     Available versions:  2.8.1 3.4.2 (~)3.5.1 {android autotools baremetal bazaar clang clearcase cmake cvs debug doc examples fakevim git ios madde mercurial perforce python qbs qnx subversion systemd test valgrind webkit winrt LINGUAS="cs de es fr it ja pl ru sl uk zh_CN zh_TW"}
     Installed versions:  3.6.0(04:52:28 PM 02/14/2016)(ios python -android -autotools -baremetal -bazaar -clang -clearcase -cmake -cvs -doc -git -mercurial -perforce -qbs -qnx -subversion -systemd -test -valgrind -webkit -winrt LINGUAS="-cs -de -fr -ja -pl -ru -sl -uk -zh_CN -zh_TW")
     Homepage:            http://doc.qt.io/qtcreator/
     Description:         Lightweight IDE for C++/QML development centering around Qt

and

# ll /usr/portage/dev-qt/qt-creator/
total 36K
-rw-r--r-- 1 root root 1.6K Jan 27 08:28 Manifest
drwxr-xr-x 1 root root   70 Jan 27 08:28 files/
-rw-r--r-- 1 root root 2.0K Feb 12 18:30 metadata.xml
-rw-r--r-- 1 root root 3.0K Jan 27 08:28 qt-creator-2.8.1.ebuild
-rw-r--r-- 1 root root 4.3K Jan 27 08:28 qt-creator-3.4.2.ebuild
-rw-r--r-- 1 root root 4.7K Jan 27 08:28 qt-creator-3.5.1.ebuild
-rw-r--r-- 1 root root 4.6K Jan 27 08:28 qt-creator-3.6.0.ebuild

Same problem applies to the eix ebuild itself, all versions of eix will not show up.

I am using eix version 0.31.7-r1 which I received on an update just today.

I did try to remove the eix database to get a fresh start but it made no difference.

Anyone else having this problem?

// Erik

Oleg Vinichenko · February 21, 2016

this should be fixed. please sync tree and check.

cardinal · February 20, 2016

Yes, same problem on stable with eix-0.30.11 and current with eix-0.31.7-r1.

erikr · February 20, 2016

Hi,

The bug that was filed related to the post I mentioned asked for info about the md5sum-cache.

I recon that eix are reading the md5sum-cache for portage to avoid the looooong walkthrough (as happens with overlays where it takes like forever)

According to https://github.com/vaeth/eix/issues/3 the reporter is asked to check $PORTDIR/metadata/md5-cache

When I do this I find something:

# pwd
/usr/portage/metadata/md5-cache/dev-qt
nuffsan dev-qt # ll qt-creator-*
-rw-r--r-- 1 root root 1.9K Feb  6 21:02 qt-creator-2.8.1
-rw-r--r-- 1 root root 3.1K Feb  6 21:02 qt-creator-3.4.2
-rw-r--r-- 1 root root 3.1K Feb  6 21:02 qt-creator-3.5.1

which is exactly what is reported by EIX.

When I do sync I normally use eix-sync.

I tried to run a emerge --sync but it made no improvements to the cache. According to the emerge manpage the md5sumcache is updated automatically for rsync-suers but funtoo does not use rsync.

I manually updated the md5sum-cache using:

# egencache --update dev-qt/qt-creator

and the cache is not in sync with the ebuilds:

# ll qt-creator-*
-rw-r--r-- 1 root root 1.9K Feb  6 21:02 qt-creator-2.8.1
-rw-r--r-- 1 root root 3.1K Feb  6 21:02 qt-creator-3.4.2
-rw-r--r-- 1 root root 3.1K Feb  6 21:02 qt-creator-3.5.1
-rw-r--r-- 1 root root 3.1K Feb 20 09:53 qt-creator-3.6.0

After running eix-sync again thingds seems to work properly:

# eix qt-creator
[I] dev-qt/qt-creator
     Available versions:  2.8.1 3.4.2 (~)3.5.1 (~)3.6.0 {android autotools baremetal bazaar clang clearcase cmake cvs debug doc examples fakevim git ios madde mercurial perforce python qbs qnx subversion systemd test valgrind webkit winrt LINGUAS="cs de es fr it ja pl ru sl uk zh_CN zh_TW"}
     Installed versions:  3.6.0(04:52:28 PM 02/14/2016)(ios python -android -autotools -baremetal -bazaar -clang -clearcase -cmake -cvs -doc -git -mercurial -perforce -qbs -qnx -subversion -systemd -test -valgrind -webkit -winrt LINGUAS="-cs -de -fr -ja -pl -ru -sl -uk -zh_CN -zh_TW")
     Homepage:            http://doc.qt.io/qtcreator/
     Description:         Lightweight IDE for C++/QML development centering around Qt

Thing is that files in the cache seems to be located in the git repository that is fecthed during emerge --sync and from that perspective things work the same as with rsync users (i.e. gentoo).

I think we need to rephrase the question; why is the md5cache not updated reflecting the ebuilds?

erikr · February 20, 2016

Posted a bug report targeting the md5sum-cache.

erikr · February 20, 2016

Just crossed my mind;

I would not suggest anyone to start updating the md5sum-cache locally.

I might be wrong but funtoo uses a GIT repository. Unlike gentoo's rsync repository GIT is aware of both server and local changes to the repository. Changes made to the file system and especially adding files that we can assume be added centrally once this is fixed may end up in conflicts that may remain a long time. I do not know if emerge --sync is prepared for that or if you will need to resolve any conflicts manually.

I expect that Gentoo rsync method would most likely detect that the server version is newer than the local and write it over, or retain the local file as that is newer. Upon later syncs the server version would be newer and the local one overwritten.

I deleted the qt-creator cache file I created during fault finding above and verified that I have no local changes to the repository:

# git status
On branch funtoo.org
Your branch is up-to-date with 'origin/funtoo.org'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        dev-qt/qt-creator-3.6.0

nothing added to commit but untracked files present (use "git add" to track)

# rm dev-qt/qt-creator-3.6.0

# git status
On branch funtoo.org
Your branch is up-to-date with 'origin/funtoo.org'.
nothing to commit, working directory clean

Oleg Vinichenko · February 20, 2016

yep, rsync is not used but before tree sync happens, cache is generated for entire tree. each time. and users getting the cache. Its not a problem. A problem could be that cache is wrong. Need to check this. To verify my idea, what you could is (and report results)

rm -rf /usr/portage

emerge --sync

du -sh /usr/portage/metadata/md5-cache

mv /usr/portage/metadata/md5-cache /tmp

egencache --update --repo=gentoo --jobs=X (where X is your CPU naumber)

du -sh /usr/portage/metadata/md5-cache

also you can do a diff between qt-creater, freshly generated and the one from /tmp/md5-cache (moved from vanilla tree), See if this differs

Oleg Vinichenko · February 20, 2016

egencache can take some time

erikr · February 20, 2016

So, based on a fresh portage (remove /usr/portage and emerge --sync) the md5-cache size reports 165M:

# du -sh /usr/portage/metadata/md5-cache
165M    /usr/portage/metadata/md5-cache

This is the same size as I had before deletion.

erikr · February 20, 2016

When running the egencache I got an error message:

# egencache --update --repo=gentoo --jobs=8
/usr/portage/eclass/depend.apache.eclass: line 129: get_libdir: command not found

The diff between the new cache and the stored cache seems to match:

# diff -r md5-cache /tmp/md5-cache/
#

The impression I get is that the process fails due to the /usr/portage/eclass/depend.apache.eclass.

# ll md5-cache/dev-qt/qt-creator-*
-rw-r--r-- 1 root root 1.9K Feb  6 21:02 md5-cache/dev-qt/qt-creator-2.8.1
-rw-r--r-- 1 root root 3.1K Feb  6 21:02 md5-cache/dev-qt/qt-creator-3.4.2
-rw-r--r-- 1 root root 3.1K Feb  6 21:02 md5-cache/dev-qt/qt-creator-3.5.1
nuffsan metadata # ll /tmp/md5-cache/dev-qt/qt-creator-*
-rw-r--r-- 1 root root 1.9K Feb 20 13:44 /tmp/md5-cache/dev-qt/qt-creator-2.8.1
-rw-r--r-- 1 root root 3.1K Feb 20 13:44 /tmp/md5-cache/dev-qt/qt-creator-3.4.2
-rw-r--r-- 1 root root 3.1K Feb 20 13:44 /tmp/md5-cache/dev-qt/qt-creator-3.5.1
# diff -r md5-cache/dev-qt/qt-creator-*  /tmp/md5-cache/dev-qt/
diff: extra operand 'md5-cache/dev-qt/qt-creator-3.5.1'
diff: Try 'diff --help' for more information.
# diff md5-cache/dev-qt/qt-creator-2.8.1  /tmp/md5-cache/dev-qt/qt-creator-2.8.1
# diff md5-cache/dev-qt/qt-creator-3.4.2 /tmp/md5-cache/dev-qt/qt-creator-3.4.2
# diff md5-cache/dev-qt/qt-creator-3.5.1  /tmp/md5-cache/dev-qt/qt-creator-3.5.1
#

And finally the size of the newly generated md5-cache:

# du -sh md5-cache/
165M    md5-cache/

erikr · February 20, 2016

I tried to replace the failing macro with absolute path (/usr/lib) and then to rerun egencache.

This time it did not break due to error and there where a lot of differences in the diff (file attached to the bug report).

The size of the cache was bigger:

# du -sh md5-cache/
167M    md5-cache/

and after an eix-sync I get the expected result from eix:

# eix qt-creator
[I] dev-qt/qt-creator
     Available versions:  2.8.1 3.4.2 (~)3.5.1 (~)3.6.0 {android autotools baremetal bazaar clang clearcase cmake cvs debug doc examples fakevim git ios madde mercurial perforce python qbs qnx subversion systemd test valgrind webkit winrt LINGUAS="cs de es fr it ja pl ru sl uk zh_CN zh_TW"}
     Installed versions:  3.6.0(04:52:28 PM 02/14/2016)(ios python -android -autotools -baremetal -bazaar -clang -clearcase -cmake -cvs -doc -git -mercurial -perforce -qbs -qnx -subversion -systemd -test -valgrind -webkit -winrt LINGUAS="-cs -de -fr -ja -pl -ru -sl -uk -zh_CN -zh_TW")
     Homepage:            http://doc.qt.io/qtcreator/
     Description:         Lightweight IDE for C++/QML development centering around Qt
# eix eix
[I] app-portage/eix
     Available versions:  0.30.4 (~)0.30.10 0.30.11 (~)0.31.4 (~)0.31.6 (~)0.31.7 (~)0.31.7-r1 {debug +dep doc nls optimization +required-use security sqlite strong-optimization strong-security swap-remote tools LINGUAS="de ru"}
     Installed versions:  0.31.7-r1(07:55:07 PM 02/19/2016)(dep nls required-use -debug -doc -optimization -security -sqlite -strong-optimization -strong-security -swap-remote -tools LINGUAS="-de -ru")
     Homepage:            https://github.com/vaeth/eix/
     Description:         Search and query ebuilds, portage incl. local settings, ext. overlays and more

Oleg Vinichenko · February 20, 2016

thx. now problem identified. will fix soon

cardinal · February 20, 2016

Merge_utils.py is a complex script : http://git.funtoo.org/funtoo-overlay/tree/funtoo/scripts/merge_utils.py

md5-cache is stale because GenCache failed.

Oleg Vinichenko · February 20, 2016

it is not stale. it does not include cache for ebuilds with EAPI=6

cardinal · February 20, 2016

Thank you for the correction.

erikr · February 21, 2016

Eix is happy and comparing the the cache with all ebuilds seems to match close enought. Only some meta data packages with identical names differs now but eix seems to catch them anyway.

Thanks Oleg.

eix behaives strage

Question

Link to comment

Share on other sites

15 answers to this question

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites