How would I automate testing a system?

[walterw]

With all of the tools such as docker and virtualbox, I can easily launch a VM, but what I would like to do is to simulate all of the typical activity I perform on my machine and replay that against the VM to see what works and doesn't work.

 

Is this possible using off-the-shelf software?  For example, I would like to plug in a USB thumbdrive, join a wifi network, have certain displays connected, etc.

[palica]

I would say something like

• https://github.com/ohwgiles/laminar
• LXD
• and a bash script with all the stuff you want to test

[walterw]

That doesn't answer how I would test a linux image (where an image is an ISO ...).  Sure, whatever I ultimately would invoke could be initiated by this, but I'm looking for the tool that would simulate network traffic, someone logging in, plugging in a monitor, etc.

I would imagine I could hack together a bunch of scripts to do stuff in virtualbox, but that is sort of limited I would imagine and tied to virtualbox.  Even then, I'd probably have to code in C or something.

[palica]

 

4 hours ago, walterw said:

With all of the tools such as docker and virtualbox, I can easily launch a VM, but what I would like to do is to simulate all of the typical activity I perform on my machine and replay that against the VM to see what works and doesn't work.

 

Is this possible using off-the-shelf software?  For example, I would like to plug in a USB thumbdrive, join a wifi network, have certain displays connected, etc.

Well sorry then, but none of your text does say "image" or "iso-imager" or anything similar. I thought your are talking about funtoo and funtoo as such can be installed of a live rescue cd, but using stages. Stages as such can be tested using LXD and simple scripts. What are you trying to solve?

Network traffic = installing packages inside the stage would generate traffic

hardware stuff is not easy to simulate would be my guess

But I still don't understand why? Do you have bad experience? Is this stuff not working? (logging as a user, plugging a monitor)

[walterw]

Well, yes, stuff generally works; however, when I need to print or something I haven't done in a year, it may not work and chasing down the problem when I need it to work is bothersome.  And, by then it would be nearly impossible to see what change may have caused it.  It might turn out that an application configuration needs to be changed.

 

If all of that stuff could be simulated, then once I have the activity captured where I could replay it later, then I would have a guarantee that a new build is good before I actually decide to reboot to it.  It is the same reason we have kits.

[palica]

Fair point, but don't know how to automate stuff such as printing - how would you tell the script the print succeeded? And do you really want to print everyday a test page? Just an example.

Sorry I can't help. Maybe someone else has an idea how to test general user interaction with the system.

Stuff gets tested by users running kits. If a bug is there it gets reported and gets squashed the best thing to do is tell your friends to use funtoo so the userbase grows and more people run software and report bugs.

I haven't had problems with:

• usb drives not being recognized by the system
• monitors being plugged in
• loggin in

Funtoo makes it easy with debian-kernel (most of the stuff just works - hardware), curated kits where stuff is not upgraded just because there is a new version of something ...

If you think of something, please let me know.

take care

[walterw]

With the advent of virtualization, I wouldn't actually do these things, they would be simulated.  I mean, there is no way Google or ... Microsoft manually test their products.  Maybe there is some proprietary product that does this testing.

It isn't whether I have issues with these things, it is whether I *COULD* have issues with these things and not know about it.  I'd rather know when I build my system that something is broken so I could just not upgrade.