xf86-video-vmware segfault

[cbdougla]

I've been having a problem recently with my Funtoo VM (VMware for Windows host).

When I run startx with the vmware video driver installed, I get a segfault such as the following:

[   326.599] (II) vmware(0): Initialized VMWARE_CTRL extension version 0.2
[   326.599] (II) vmware(0): Initialized VMware Xinerama extension.
[   326.599] (II) vmware(0): vgaHWGetIOBase: hwp->IOBase is 0x03d0
[   326.600] (EE) vmware(0): Unable to map mmio BAR. Invalid argument (22)
[   326.607] (EE) vmware(0): Unable to map frame buffer BAR. Invalid argument (22)
[   326.607] (EE)
[   326.607] (EE) Backtrace:
[   326.607] (EE) 0: Xorg (xorg_backtrace+0x4d) [0x555a895894cd]
[   326.607] (EE) 1: Xorg (0x555a893e3000+0x1aa199) [0x555a8958d199]
[   326.607] (EE) 2: /lib64/libpthread.so.0 (0x7f61c95c4000+0x14ab0) [0x7f61c95d8ab0]
[   326.607] (EE) 3: /lib64/libc.so.6 (0x7f61c93fa000+0x16a82d) [0x7f61c956482d]
[   326.607] (EE) 4: /usr/lib64/xorg/modules/drivers/vmware_drv.so (0x7f61ca984000+0x6e9b) [0x7f61ca98ae9b]
[   326.607] (EE) 5: Xorg (AddScreen+0xd7) [0x555a8943a417]
[   326.608] (EE) 6: Xorg (InitOutput+0x3fe) [0x555a89479e6e]
[   326.608] (EE) 7: Xorg (0x555a893e3000+0x5af7f) [0x555a8943df7f]
[   326.608] (EE) 8: /lib64/libc.so.6 (__libc_start_main+0xe7) [0x7f61c941bbe7]
[   326.608] (EE) 9: Xorg (_start+0x2a) [0x555a89427dea]
[   326.608] (EE)
[   326.608] (EE) Segmentation fault at address 0x0
[   326.608] (EE)
Fatal server error:
[   326.608] (EE) Caught signal 11 (Segmentation fault). Server aborting
[   326.608] (EE)
[   326.608] (EE)
Please consult the The X.Org Foundation support
         at http://wiki.x.org
 for help.
[   326.608] (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information.
[   326.608] (EE)
[   326.613] (EE) Server terminated with error (1). Closing log file.

 

If I uninstall xf86-video-vmware or just move the driver file to a different name, the X11 starts up and uses the modeset driver and seems to work ok.  

Because I can get it to work with modeset, this is really a minor issue but it bugs me that I can't use the vmware driver.  I've searched several times for hints on how to fix this but I am at a loss.

I've tried various kernel versions (4.19.2-gentoo, 4.20.0-gentoo, 4.9.168_p1 from debian LTS and 5.0.0-pf2) and I've played around with USE flags to the point that I don't remember everything I have done.

 

In addition, I followed an article I found on how to get a good stacktrace so my system is currently built with the FEATURES="splitdebug" variable set in make.conf.

So I've got a full gdb backtrace also which I will post at the end.

 

Any advice would be greatly appreciated.  I am definitely in over my head on this problem.  Again, it's a minor issue.

 

 

gdb-bt-full.txt

[skyark]

Hi, did you make a xorg.conf file?

Can you paste the output.

lspci -k

and the 

emerge --info

please...

[cbdougla]

21 hours ago, skyark said:

Hi, did you make a xorg.conf file?

Can you paste the output.

lspci -k

and the 

emerge --info

please...

I should have included that information in the original post.  Sorry about that.

I have not created an xorg.conf file.  I am letting it autodetect.

 

I am attaching text files with the information requested.

 

 

emerge-info.txt lspci.txt

[skyark]

39 minutes ago, cbdougla said:

I should have included that information in the original post.  Sorry about that.

I have not created an xorg.conf file.  I am letting it autodetect.

 

I am attaching text files with the information requested.

 

 

emerge-info.txt 7.97 kB · 1 download lspci.txt 5.52 kB · 0 downloads

Have you include this in your make.conf file before?

VIDEO_CARDS="vmware"
INPUT_DEVICES="evdev keyboard mouse synaptics vmmouse"

If not doit then update your system whit your new make file.

emerge -auvDN --with-bdeps=y @world

And we'll see how it goes.

[cbdougla]

2 hours ago, skyark said:

Have you include this in your make.conf file before?

VIDEO_CARDS="vmware"
INPUT_DEVICES="evdev keyboard mouse synaptics vmmouse"

If not doit then update your system whit your new make file.

emerge -auvDN --with-bdeps=y @world

And we'll see how it goes.

I remember doing something like that but I've tried so many things that I can't remember.

 

Just to be clear, it uses the vmware driver...or it tries to and then segfaults .  If I move the vmware driver out of the way, it uses modeset and works fine.

I added those lines and did an emerge --newuse --deep --with-bdeps=y -q -j3 @world.

It rebuilt x11-drivers/xf86-input-vmmouse-13.1.0-r1 to and x11-base/xorg-drivers-1.20.  I then rebooted and tried startx again with the same results.

Thank you for the suggestion. 

[cbdougla]

well, I have an update to this.

I downloaded and compiled the xf86-video-vmware driver from the xorg gitlab repo (https://gitlab.freedesktop.org/xorg/driver/xf86-video-vmware) and copied the resulting driver into the /usr/lib/xorg/modules/drivers folder and it seems to be working.

Both drivers report to be version 13.3.0.  There still seems to be some strange behavior (it's crashed a couple of times on startup because it couldn't find an input driver?).

Very odd.  

Xorg.0.log

[drobbins]

@cbdougla I am also experiencing this in a vCenter environment, but it's only happening on one VM, so it's very odd. I am still investigating. It definitely doesn't happen all the time. display settings on each VM are "auto" (which is normal for vCenter).