You are not logged in.

#1 2012-11-13 12:24:27

gevious
Member
Registered: 2012-10-23
Posts: 10

[SOLVED] Missing python modules

Hi,

I've recently realised that I'm missing a number of 'out the box' python modules. Most notably sqlite, pyexpat, readline, _elementtree and others. This prevents me from building some packages and is a general nuisance. I've tried rebuilding python a number of times, but the modules aren't there.

Its worth noting that I didn't always have this problem. I can't remember exactly when it happened. Possibly when I removed python 3, or emerged world after a clean install. Either way, I haven't found anybody with this issue and hoping that somebody can point me in the right direction.

emerge --info

Portage 2.3.4-r9 (funtoo/1.0/linux-gnu/arch/x86-64bit, gcc-4.6.3, glibc-2.15-r3, 3.6.6 x86_64)
=================================================================
System uname: Linux-3.6.6-x86_64-Intel-R-_Core-TM-_i7-3610QM_CPU_@_2.30GHz-with-gentoo-2.2.0
Timestamp of tree: Thu, 08 Nov 2012 06:45:01 +0000
app-shells/bash:          4.2_p37
dev-java/java-config:     2.1.12-r1000
dev-lang/python:          2.7.3-r1000
dev-util/cmake:           2.8.9-r1
sys-apps/baselayout:      2.2.0-r3
sys-apps/openrc:          0.10.2-r6
sys-apps/sandbox:         2.6
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6
sys-devel/binutils:       2.22-r1
sys-devel/gcc:            4.6.3
sys-devel/gcc-config:     1.5-r1
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.4-r2 (virtual/os-headers)
sys-libs/glibc:           2.15-r3
Repositories: gentoo
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=corei7 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-march=corei7 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parse-eapi-ebuild-head preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--sort-common -Wl,--as-needed"
LINGUAS="en"
MAKEOPTS="-j17"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="git://github.com/funtoo/ports-2012.git"
SYNC_USER="root"
USE="32bit X acl acpi alsa amd64 bash-completion berkdb build bzip2 cdr cracklib crypt cxx dbus dri dvd dvdr dvdread flac gdbm gif gpm iconv icu ipv6 jpeg lame mad mmx modules mp3 mpeg mudflap multilib ncurses nls nptl nvidia ogg opengl openmp pcre png pppd python qt4 readline resolvconf sound sse sse2 symlink tcpd tiff truetype udev unicode usb vorbis wavpack wifi xml xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias authn_core authz_core socache_shmcb unixd" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="evdev keyboard mouse synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" NGINX_MODULES_HTTP="uwsgi memcached access browser gzip proxy rewrite" PHP_TARGETS="php5-3" PYTHON_ABIS="2.7" PYTHON_TARGETS="python2_7" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19" USERLAND="GNU" VIDEO_CARDS="intel nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, SYNC_UMASK

Last edited by gevious (2012-11-16 06:28:34)

Offline

#2 2012-11-14 01:08:26

overkill
Member
Registered: 2011-09-13
Posts: 243

Re: [SOLVED] Missing python modules

Have you tried `'python-updater`?

# python-updater --help
python-updater -- Find & rebuild packages broken due to a Python upgrade

Usage: python-updater [OPTION]

Options:
    -h, --help      Print usage
    -V, --version   Print version
    -p, --pretend   Pretend (don't do anything)
    -v, --verbose   Increase verbosity (may be specified multiple times)
    --reinstall-identical-versions
                    Reinstall identical versions of packages
    -P PM, --package-manager PM
                    Use package manager PM, where PM can be one of:
		    * portage (Default)
		    * pkgcore
		    * paludis
    -c CMD, --command CMD
                    Pipe found packages to command CMD instead of invoking package
                    manager. Only for debug and script use.
    --package-manager-command CMD
                    Call CMD instead of the default command for the specified
                    package manager.
    -eCHECK --enable-CHECK
                    Enable CHECK where CHECK can be one of:
                    * all
                    * manual         (Enabled by default)
                    * need_rebuild   (Enabled by default)
                    * pylibdir       (Enabled by default)
                    * PYTHON_ABIS    (Enabled by default)
                    * shared_linking (Enabled by default)
                    * static_linking (Enabled by default)
    -dCHECK --disable-CHECK
                    Disable CHECK. See --enable option for a list of checks
    -- OPTIONS      Pass additional options to package manager

See CHECKS section in the manpage for explanations about checks and
EXAMPLES section to learn how to use them.

You could start with `python-updater --pretend` and see what if finds.  It can take some time, even on a fast machine.

# python-updater -p
 * Starting Python Updater...
 * Main active version of Python:    2.7
 * Active version of Python 2:       2.7
 * Active version of Python 3:       3.2
 * Globally supported Python ABIs in installed repositories:
 *   gentoo:                         2.5 2.6 2.7 3.1 3.2 3.3 2.5-jython 2.7-jython 2.7-pypy-1.8 2.7-pypy-1.9
 *   localhost:                      2.5 2.6 2.7 3.1 3.2 3.3 2.5-jython 2.7-jython 2.7-pypy-1.8 2.7-pypy-1.9
 * No packages need to be reinstalled.

You probably should have just left python 3 there and set default to 2.7 in make.conf.  If I were you, I'd find the funtoo python docs and restore python-3.2  There was a recent portage news (eselect news list) item about the new python system.

Offline

#3 2012-11-14 01:18:30

overkill
Member
Registered: 2011-09-13
Posts: 243

Re: [SOLVED] Missing python modules

Here's the news item from eselect:
# eselect news read 10
2012-11-06-PYTHON_TARGETS-deployment
  Title                     PYTHON_TARGETS deployment
  Author                    Michał Górny <mgorny@gentoo.org>
  Posted                    2012-11-06
  Revision                  1

Recently, a few new Python eclasses have been deployed. As ebuilds
migrate, the way they support multiple Python implementations will
change. The previous method built Python modules for Python
implementations selected through `eselect python'. The new method uses
the PYTHON_TARGETS USE flags to explicitly name the implementations the
modules shall be built for.

If you are running a modern system with only Python 2.7 & 3.2 installed,
then you don't have to do anything. The defaults will simply fit you,
and let you keep your system up-to-date when new Python versions are
deployed.

However, if you'd like to use another set of Python implementations, you
will need to set PYTHON_TARGETS in your make.conf file appropriately.
This variable names the enabled implementations in the standard way
common to all USE_EXPAND variables.

For example, a setup enabling all major Python implementations would
look like:

    PYTHON_TARGETS="python2_7 python3_2 pypy1_9 jython2_5"

The variable should list all Python implementations which are going to
be used on the system; missing a particular value there will result
in missing Python modules.

A complete list of all possible values can be obtained using a command
equivalent to the following:

    emerge -1pv dev-python/python-exec

For more details, please see the python-r1 User's Guide [1].

[1] http://www.gentoo.org/proj/en/Python/py … -guide.xml

Offline

#4 2012-11-14 05:26:39

angry_vincent
Staff
From: Ukraine
Registered: 2010-10-07
Posts: 687

Re: [SOLVED] Missing python modules

this one has no impact to Funtoo. Funtoo merged progress overlay advanced python eclass and forked ebuilds long time ago.
@gevoius, normally you should have python-3 installed, it is present in stage3, i see no point in removing it.

Offline

#5 2012-11-14 11:05:43

gevious
Member
Registered: 2012-10-23
Posts: 10

Re: [SOLVED] Missing python modules

Thank you for your help. I've reinstalled python 3.2 and added it to the PYTHON_TARGETS and PYTHON_ABIS

however, still no luck. even after rebuilding both python 2.7 and 3.2 and using the python updater, still no readline library (and others). Is there perhaps a global USE flag I'm missing?

# These settings were set by the metro build script that automatically built this stage.
# Please consult /etc/make.conf.example for a more detailed example.

CFLAGS="-march=corei7 -O2 -pipe"
CXXFLAGS="-march=corei7 -O2 -pipe"
#CPPFLAGS="-I/usr/include/simgear -I/usr/include" LDFLAGS="-L/usr/lib/simgear -L/usr/lib64 -Lusr/lib"


MAKEOPTS="-j17"
VIDEO_CARDS="intel nvidia"
INPUT_DEVICES="evdev keyboard mouse synaptics"

LINGUAS="en"
ALSA_CARDS="hda-intel"
PYTHON_ABIS="2.7 3.2"
PYTHON_TARGETS="python2_7 python3_2"
RUBY_TARGETS="ruby19"
KERNEL="build symlink"
GRUB_PLATFORMS="efi-64"
NGINX_MODULES_HTTP="uwsgi memcached access browser gzip proxy rewrite"
USE="${KERNEL} 32bit X acpi bash-completion dbus opengl sound symlink usb wifi
     png gif nvidia alsa qt4 python -ssl -cups -accessibility -jack -pam"

Offline

#6 2012-11-14 12:52:28

angry_vincent
Staff
From: Ukraine
Registered: 2010-10-07
Posts: 687

Re: [SOLVED] Missing python modules

remove PYTHON_TARGETS, rebuild all python-dependant packages. Show eselect profile list (you do not need PYTHON_ABIS, it is set by profile).

Offline

#7 2012-11-15 08:37:07

gevious
Member
Registered: 2012-10-23
Posts: 10

Re: [SOLVED] Missing python modules

I've removed both PYTHON_ABIS and PYTHON_TARGETS from my make.conf. I've tried to get a list of packages using python, but get an error:

gevious@localhost ~ $ equery uses python
Traceback (most recent call last):
  File "/usr/bin/equery", line 38, in <module>
    equery.main(sys.argv)
  File "/usr/lib64/python2.7/site-packages/gentoolkit/equery/__init__.py", line 354, in main
    loaded_module.main(module_args)
  File "/usr/lib64/python2.7/site-packages/gentoolkit/equery/uses.py", line 292, in main
    output = get_output_descriptions(pkg, global_usedesc)
  File "/usr/lib64/python2.7/site-packages/gentoolkit/equery/uses.py", line 177, in get_output_descriptions
    if pkg.metadata is None:
  File "/usr/lib64/python2.7/site-packages/gentoolkit/package.py", line 121, in metadata
    from gentoolkit.metadata import MetaData
  File "/usr/lib64/python2.7/site-packages/gentoolkit/metadata.py", line 45, in <module>
    import xml.etree.cElementTree as etree
  File "/usr/lib64/python2.7/xml/etree/cElementTree.py", line 3, in <module>
    from _elementtree import *
ImportError: No module named _elementtree

This is part of the problem (since some python libs are missing). I've rebuild readline and ncurses then python, but still don't have the readline module for python.

Is there another command I could use? I'm not exactly sure how to "rebuild all python-dependant packages"

Offline

#8 2012-11-15 08:46:34

angry_vincent
Staff
From: Ukraine
Registered: 2010-10-07
Posts: 687

Re: [SOLVED] Missing python modules

# emerge -1 python:2.7 python:3.2 
# emerge -1 expat 
# python-updater
# revdep-rebuild

Notice, that you must have ncurses, readline and sqlite flags turned on in both pythons.

Offline

#9 2012-11-15 10:12:54

gevious
Member
Registered: 2012-10-23
Posts: 10

Re: [SOLVED] Missing python modules

I did all those commands, but nothing has changed. Python updater didn't find anything to update.

Offline

#10 2012-11-15 10:36:53

angry_vincent
Staff
From: Ukraine
Registered: 2010-10-07
Posts: 687

Re: [SOLVED] Missing python modules

what make you think above listed modules missing?

Offline

#11 2012-11-15 12:13:34

gevious
Member
Registered: 2012-10-23
Posts: 10

Re: [SOLVED] Missing python modules

if I type help('modules') in my python shell, they're not listed. Also, as per the post above, module named _elementtree is missing. In addition to this, I used to have sqlite and readline on my machine, but then one day they weren't there. In my day job I do python development, and one day my test cases failed, which made me realise that something was wrong (because they couldn't connect to sqlite). My feeling is that for some reason the USE cases aren't actually working as they should (ie it says its using readline, but for some reason its not). I'm very new to gentoo, so am not sure how things work internally (yet).

Offline

#12 2012-11-15 12:34:33

angry_vincent
Staff
From: Ukraine
Registered: 2010-10-07
Posts: 687

Re: [SOLVED] Missing python modules

show emerge -pv python

Offline

#13 2012-11-15 15:15:46

angry_vincent
Staff
From: Ukraine
Registered: 2010-10-07
Posts: 687

Re: [SOLVED] Missing python modules

OK, unforseen USE flag "build" is enabled, do not use it (disable). It is a special flag that used during stage generation, internal only, never turn it intentionally. This flag disables many modules.

Offline

#14 2012-11-16 06:28:17

gevious
Member
Registered: 2012-10-23
Posts: 10

Re: [SOLVED] Missing python modules

WHOOHOO. That did the trick. I removed build, and rebuilt and the modules are back. Thanks so much, angry_vincent.

Offline

Board footer

Powered by FluxBB