Jump to content
funtoo forums
palica

Anonymous usage reports for funtoo kits usage

Recommended Posts

Like you, I was able to send after changing report.conf:

# To report profiles information
# the same as epro show-json
# 2018-02-02 dlc : Disabled profile-info per palica in
# https://forums.funtoo.org/topic/1382-anonymous-usage-reports-for-funtoo-kits-usage/
# to hopefully get send to work:
# profile-info:y
profile-info:n

Interestingly, ./report show-json now outputs nothing using either version of report.conf.

Share this post


Link to post
Share on other sites

I'm still having no luck getting the profile info portion of the report to submit to elastic search. I set the cpu-info report to count the number of "processors" and report that. let me know if these numbers look more accurate.

Share this post


Link to post
Share on other sites
3 hours ago, dlc said:

Interestingly, ./report show-json now outputs nothing using either version of report.conf.

Could you paste your entire /etc/report.conf file? Typically if it doesn't report anything, that's because it didn't find what it expected in the conf file.

Share this post


Link to post
Share on other sites

Sure, here's the original, which should be identical to the README.txt version save for the added UUID:

## Configuration for selecting and deselecting which data
## is reported by the funtoo anonymous reporting tool
##
## All options are defaulted to report, you can change an item
## by altering the "y" and "n" to indicate either yes (y) report it
## or no (n) do not report it.
##

# To report cpu info which includes clock speed, model name,
# and cpu cores
cpu-info:y

# To report memory info which includes the amount of free memory,
# the amount of memory available, total amount of swap space,
# and the amount of free swap space
mem-info:y

# To report kernel info including O.S. type, release and version
kernel-info:y

# Allows the reporter to search your /boot directory and list
# any kernels it finds
# (limited to kernel names that start with "kernel" or "vmlinuz")
boot-dir-info:y

# To report versions of key softwares on your system including
# portage, ego, python, gcc, and glibc
version-info:y

# To report the contents of /var/lib/portage/world
world-info:y

# To report profiles information
# the same as epro show-json
profile-info:y

# To report kit versions as reported by ego
# extracted from ego kit show
kit-info:y

# A unique identifier for this reporting machine 
UUID:8f642ea9-9dc2-4a6c-93e3-d6ebb57cd4b1 

 

Share this post


Link to post
Share on other sites

This is really starting to come together.. the tool should be reporting nicely now if you have the latest commit. Now we really need to think about any new functionality we want out of the reporting tool and the quality of the data it produces. Also any information we would like to add to the report.

Share this post


Link to post
Share on other sites

considering this comment on #funtoo IRC channel:
 

<@Azerthoth> haxmeister, I would add a few more common points of problems, video card family and class, 
                   driver. Sound system, drivers and controllers (alsa/oss/pulse/etc)

Some of this sounds good.. especially vid card info

Share this post


Link to post
Share on other sites
[Request] ** [http://elk.liguros.net:9200]-[400] [mapper_parsing_exception] failed to parse [timestamp], called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at ./report line 45. With vars: {'request' => {'body' => {'profile-info' => {'flavor' => ['desktop'],'mix-ins' => ['no-systemd','xfce','mediadevice-audio-pro','X','audio','print'],'arch' => ['x86-64bit'],'subarch' => ['core2_64'],'build' => ['current']},'timestamp' => 'Wed Feb  7 01:10:32 2018','version-info' => {'python versions' => ['2.7.13-r1','3.4.6-r1'],'ego version' => '2.3.3-r1','gcc versions' => ['5.4.0'],'portage version' => '2.3.18-r2'},'kit-info' => {'games-kit' => 'master','xorg-kit' => '1.19-primt','text-kit' => 'master','core-hw-kit' => 'master','media-kit' => '1.1-prime','haskell-kit' => '1.1-prime','ruby-kit' => '1.1-prime','ml-lang-kit' => '1.1-prime','xfce-kit' => '4.12-prime','dev-kit' => '1.1-prime','security-kit' => '1.0-prime','core-kit' => '1.0-prime','php-kit' => 'master','net-kit' => 'master','editors-kit' => 'master','perl-kit' => '5.24-prime','lisp-scheme-kit' => '1.1-prime','nokit' => 'master','lang-kit' => '1.1-prime','java-kit' => '1.1-prime','kde-kit' => '5.10-prime','gnome-kit' => '3.20-prime','desktop-kit' => '1.1-prime','science-kit' => 'master','python-kit' => '3.4-prime'},'kernel-info' => {'ostype' => 'Linux','version' => '#1 SMP Tue Jan 23 04:26:34 -00 2018','osrelease' => '4.14.12-2'},'cpu-info' => {'cpu MHz' => '2000','flags' => ['fpu','vme','de','pse','tsc','msr','pae','mce','cx8','apic','sep','mtrr','pge','mca','cmov','pat','pse36','clflush','dts','acpi','mmx','fxsr','sse','sse2','ss','ht','tm','pbe','syscall','nx','lm','constant_tsc','arch_perfmon','pebs','bts','rep_good','nopl','cpuid','aperfmperf','pni','dtes64','monitor','ds_cpl','vmx','smx','est','tm2','ssse3','cx16','xtpr','pdcm','sse4_1','xsave','lahf_lm','pti','tpr_shadow','vnmi','flexpriority','dtherm'],'model name' => 'Intel(R) Core(TM)2 Quad CPU    Q9550  @ 2.83GHz','processors' => 4},'version' => '1.1','mem-info' => {'MemAvailable' => 1459132,'SwapTotal' => 1048568,'MemTotal' => 8170200,'MemFree' => 1213172,'SwapFree' => 561820},'world-info' => ['app-editors/vim','app-misc/screenfetch','app-office/libreoffice','app-portage/eix','app-text/evince','media-gfx/fbgrab','media-gfx/gimp','media-libs/bio2jack','media-libs/fluidsynth-dssi','media-libs/gexiv2','media-plugins/alsa-plugins','media-sound/alsa-tools','media-sound/alsa-utils','media-sound/alsamixergui','media-sound/alsaplayer','media-sound/ardour','media-sound/audacity','media-sound/fluid-soundfont','media-sound/fluidsynth','media-sound/hydrogen','media-sound/musescore','media-sound/qjackctl','media-sound/sox','media-video/qt-recordmydesktop','media-video/recordmydesktop','media-video/simplescreenrecorder','media-video/vlc','net-irc/hexchat','sys-kernel/debian-sources','sys-kernel/linux-firmware','www-client/chromium','www-client/netsurf','x11-apps/xinit','x11-base/xorg-x11','x11-misc/shutterbug','xfce-base/xfce4-meta','xfce-extra/xfce4-alsa-plugin','xfce-extra/xfce4-cpufreq-plugin','xfce-extra/xfce4-cpugraph-plugin','xfce-extra/xfce4-diskperf-plugin','xfce-extra/xfce4-netload-plugin'],'boot-dir-info' => {'available kernels' => ['vmlinuz-4.8.0-53-generic','kernel-debian-sources-x86_64-4.14.12-2','vmlinuz-4.9.6_iso']}},'path' => '/funtoo/user/7d8bccd8-4baf-495e-a05a-d512b5e61018','mime_type' => 'application/json','method' => 'POST','serialize' => 'std','qs' => {},'ignore' => []},'status_code' => 400,'body' => {'status' => 400,'error' => {'type' => 'mapper_parsing_exception','root_cause' => [{'type' => 'mapper_parsing_exception','reason' => 'failed to parse [timestamp]'}],'reason' => 'failed to parse [timestamp]','caused_by' => {'type' => 'illegal_argument_exception','reason' => 'Invalid format: "Wed Feb  7 01:10:32 2018"'}}}}

This is all I can get out of send using

-rwxr-xr-x 1 root root 2.9K Feb  5 20:57 report

However, show-json produces results every time now.

 

Edit:  retrieved zip again getting

-rwxr-xr-x 1 root root 2.9K Feb  6 18:01 report

and all seems nominal now!

Edited by dlc
New version did not misbehave

Share this post


Link to post
Share on other sites

This was due to data structure change on the server-side the latest git code should work as we changed the timestamp format being submitted. So for now running latest reporter version is kind of required as we figure things out. Sorry for this.

Share this post


Link to post
Share on other sites

After changing timestamp we now can track historical data

image.png

For example yesterday only 5 systems reported:

image.png

Today you can see 7 systems:

image.png

So we can see in time what versions and what kits were used on the systems that are reporting.

 

Share this post


Link to post
Share on other sites
13 hours ago, Oleg Vinichenko said:

what is merged?

20 hours ago, palica said:

... pull requests and merge them ...

thank you @haxmeister

The ebuilds for Search::Elasticsearch and for Funtoo::Report (the tool to submit info) have been committed to the funtoo tree.

Share this post


Link to post
Share on other sites

With the latest tree update there is a package for reporting usage of funtoo profiles, kits and other stuff. It is called -

dev-perl/Funtoo-Report

Please test it out and start reporting your machine to the elasticsearch backend. The kibana frontend can be found on this address http://elk.liguros.net/, I have made a simple Dashboard (http://elk.liguros.net/app/kibana#/dashboard/41e03450-0b28-11e8-a9dd-1718169924ab?_g=(refreshInterval%3A('%24%24hashKey'%3A'object%3A343'%2Cdisplay%3A'15 minutes'%2Cpause%3A!f%2Csection%3A2%2Cvalue%3A900000)%2Ctime%3A(from%3Anow-7d%2Cmode%3Aquick%2Cto%3Anow)). I and haxmeister would be happy to hear your thoughts and critique.

The source code for the tool can be found here:

https://github.com/haxmeister/funtoo-reporter.git

Here a simple howto use this tool:

e

emerge -av Funtoo-Report

Just launching the program will show you a help menu:

funtoo-report

Funtoo anonymous data reporting tool usage:

funtoo-report send              Send the report to funtoo's data collection
funtoo-report show-json         Show the output that will be sent, in JSON format
funtoo-report help              Show this help list

Output can be ommitted by modifying the /etc/report.conf file

help shows you the same output:

funtoo-report help

To see what data the report is generating use the show-json option:

funtoo-report show-json

You may get an error that no config file is found at /etc/report.conf and it will then try to create one with all available options turned on:

Could not open file the configuration file at /etc/report.conf
Attempting to create one...

A config file has been generated at /etc/report.conf
Please review this file for errors.

You can send your report to the elastic search database using the send option which will output nothing on the console if it is successful:

funtoo-report send

It is ok to create a cron job for this to automatically submit new data to ES (Elasticsearch). It should handle the load :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×