The number of popcon.debian.org-submissions is falling

classic Classic list List threaded Threaded
302 messages Options
1234 ... 16
Reply | Threaded
Open this post in threaded view
|

The number of popcon.debian.org-submissions is falling

Petter Reinholdtsen
The number of submissions to the Debian popularity-contest collector
is falling, and has done so for some time now.  This can be easily
seen on <URL: http://popcon.debian.org/stat/sub-i386.png >.

This is mostly caused by a fall in the number of Lenny installations,
as can be seen from
<URL: http://popcon.debian.org/stat/release-1year.png >.

Anyone got any idea how to can get more machines to report to
popcon.debian.org?  Or can there be some other problem causing the
fall in the number of submissions?

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100720124149.GF6444@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Jesús M. Navarro-2
Hi, Petter:

On Tuesday 20 July 2010 14:41:49 Petter Reinholdtsen wrote:

> The number of submissions to the Debian popularity-contest collector
> is falling, and has done so for some time now.  This can be easily
> seen on <URL: http://popcon.debian.org/stat/sub-i386.png >.
>
> This is mostly caused by a fall in the number of Lenny installations,
> as can be seen from
> <URL: http://popcon.debian.org/stat/release-1year.png >.
>
> Anyone got any idea how to can get more machines to report to
> popcon.debian.org?  Or can there be some other problem causing the
> fall in the number of submissions?

I for one don't allow for popcon on production machines under my control.  I'm
not going into details about why this is the case now, but that's the fact
(on a side note I'd want to know if there's any easy way to mimic/modify
Debian's server/client popcon enviro so it can be used to produce my own
internal stats).

If that's the case with other sysadmins that probably would mean that main
source of popcon machines are home systems which in turn would make
understandable for popcon numbers to decrease by the "EOL" of a Stable
release: home users, being more akeen to novelties are probably moving to
more "bleeding edge" distributions, and being home users they are probably
doing it by reformatting their systems more than by leaving old systems their
way and using a different distro for new ones.

All in all, a look at http://popcon.debian.org/stat/sub-i386.png makes me
think that:
1) Current reduction from April onwards is statisticallly non-significant with
a main trend steadly growing since Aug, 2007.
2) It is i386 the one that basically is stucked which, again, it's no wonder
since the vast majority of general-purpouse CPUs are amd64 now and, contrary
to 'etch', 'lenny' is quite an usable 64 bit system.
3) Being both 'Y' axis related to number of installs on (mainly, I expect)
general purpouse systems, I have to wonder why there are different scales for
i386 and "overall".  It takes some time to understand there are *not* as many
i386 systems as overall (with or without i386?) in the popcon and anyway it's
impossible to know which 'Y' axis belong to what arch.

Cheers.


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/201007201554.31301.jesus.navarro@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Clint Adams
In reply to this post by Petter Reinholdtsen
On Tue, Jul 20, 2010 at 02:41:49PM +0200, Petter Reinholdtsen wrote:
> Anyone got any idea how to can get more machines to report to
> popcon.debian.org?  Or can there be some other problem causing the
> fall in the number of submissions?

Everyone I have spoken with regarding this who does not report
claims to be acting out of security/privacy paranoia, so I would
suggest addressing that somehow.


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100720144712.GA20843@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Giacomo Catenazzi
> On Tue, Jul 20, 2010 at 02:41:49PM +0200, Petter Reinholdtsen wrote:
>> Anyone got any idea how to can get more machines to report to
>> popcon.debian.org?  Or can there be some other problem causing the
>> fall in the number of submissions?
>

I mount filesystem with "noatime" (at home and on my servers),
and AFAIK popcon don't work with such configuration, so I don't
install it.

I don't know if "relatime" now works, and if popcon can works
with "relatime" (or ev. if popcon can check written files)

ciao
        cate


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/4C45B906.5060109@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Petter Reinholdtsen
[Giacomo A. Catenazzi]
> I mount filesystem with "noatime" (at home and on my servers), and
> AFAIK popcon don't work with such configuration, so I don't install
> it.

There is no need to avoid popularity-contest just because you are
mounting file systems with noatime.  The set of installated packages
will be reported and this is very useful for Debian to know.  The only
thing affected by noatime is that the information on what packages
were used last week is not reported.

> I don't know if "relatime" now works, and if popcon can works with
> "relatime" (or ev. if popcon can check written files)

The popularity-contest package also work fine with relatime.

Thanks for bringing this up.  I guess we should look at the FAQ or
something to make it clear on this point.  Where did you get the idea
that popcon don't work with noatime?  Is there some documentation we
could update to avoid others getting the same idea?

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100720152653.GG6444@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Petter Reinholdtsen
In reply to this post by Jesús M. Navarro-2

[Jesús M. Navarro]
> I for one don't allow for popcon on production machines under my
> control.

Sad to hear that your machines are not able to influence the parts of
Debian which uses the popcon.debian.org numbers to improve Debian
(like the CD/DVD builds :).

> I'm not going into details about why this is the case now, but
> that's the fact (on a side note I'd want to know if there's any easy
> way to mimic/modify Debian's server/client popcon enviro so it can
> be used to produce my own internal stats).

I did this for Debian Edu (<URL: http://popcon.skolelinux.org/ >), and
the scripts to set up your own server is in the popularity-contest
package, in /usr/share/doc/popularity-contest/examples/.

The clients can be configured to submit reports to several URLs, and
you can then get it to report to your own collector or the debian
collector and your own collector (like we do with Debian Edu).

> If that's the case with other sysadmins that probably would mean
> that main source of popcon machines are home systems which in turn
> would make understandable for popcon numbers to decrease by the
> "EOL" of a Stable release:

Given the amount of server packages listed as used in
popcon.debian.org, I doubt it is a general trend. :)

> 1) Current reduction from April onwards is statisticallly
> non-significant with a main trend steadly growing since Aug, 2007.

Could be.  Hard to predict the future. :)

I suspect it is a significant reduction, but I do not know the cause.
Perhaps Debian have a reduced user base, people moving to other
distributions.  Perhaps the popcon.debian.org server is overloaded and
unable to handle more requests (unlikely, only ~14k submissions via
http every day.  Perhaps some strange rumor have spread that popcon do
not work on laptops, and more and more laptops are being deployed in
the Debian community.  Or perhaps something completely different.  I
do not know, but hope that bringing it up here might get more eyes to
look at the issue and perhaps find a way to get more popcon
submissions. :)

> 2) It is i386 the one that basically is stucked which, again, it's
> no wonder since the vast majority of general-purpouse CPUs are amd64
> now and, contrary to 'etch', 'lenny' is quite an usable 64 bit
> system.

Well, the overall count is sinking, so that would mean that the "lost"
i386 machines are not replaced with amd64 machines. :)

> 3) Being both 'Y' axis related to number of installs on (mainly, I
> expect) general purpouse systems, I have to wonder why there are
> different scales for i386 and "overall".  It takes some time to
> understand there are *not* as many i386 systems as overall (with or
> without i386?) in the popcon and anyway it's impossible to know
> which 'Y' axis belong to what arch.

Sure, the graphs could be made easier to read, but that will have to
be the topic of a different thread. :)

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/2fltynu5ffp.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Yves-Alexis Perez-2
In reply to this post by Giacomo Catenazzi
On 20/07/2010 16:56, Giacomo A. Catenazzi wrote:
> I mount filesystem with "noatime" (at home and on my servers),
> and AFAIK popcon don't work with such configuration, so I don't
> install it.

It can at least report the installed packages, I guess?

Cheers,
--
Yves-Alexis


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/4C45C3B8.6060105@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Jonathan Clarke-2
In reply to this post by Petter Reinholdtsen
On 20/07/2010 14:41, Petter Reinholdtsen wrote:

> The number of submissions to the Debian popularity-contest collector
> is falling, and has done so for some time now.  This can be easily
> seen on<URL: http://popcon.debian.org/stat/sub-i386.png>.
>
> This is mostly caused by a fall in the number of Lenny installations,
> as can be seen from
> <URL: http://popcon.debian.org/stat/release-1year.png>.
>
> Anyone got any idea how to can get more machines to report to
> popcon.debian.org?  Or can there be some other problem causing the
> fall in the number of submissions?

Speaking of my own experience: I always install it, except on test
machines where the results would be futile anyway. However, having done
this for years, I now wonder every time I see the installer question
whether there's any point, since I never see any feedback.

Maybe making results more visible could improve adoption?

A simple link to http://popcon.debian.org/ in the installer? (or is
there one already? I don't remember, sorry)

A script to display statistics about one's own system, packages used
last week, packages not used for >3 months, most used packages, popular
packages we're not using, etc?

Maybe a simple stats report every week/month/trimester, giving out
details about most popular new packages, packages "losing popularity",
top 10, etc? (as an opt-in mailing list, or similar)

Just some ideas :)

Jonathan
--
--------------------------------------------------------------
Jonathan Clarke - [hidden email]
--------------------------------------------------------------
Ldap Synchronization Connector (LSC) - http://lsc-project.org
--------------------------------------------------------------


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/4C45C540.5020102@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Bernhard R. Link-2
In reply to this post by Yves-Alexis Perez-2
* Yves-Alexis Perez <[hidden email]> [100720 17:41]:
> On 20/07/2010 16:56, Giacomo A. Catenazzi wrote:
> > I mount filesystem with "noatime" (at home and on my servers),
> > and AFAIK popcon don't work with such configuration, so I don't
> > install it.
>
> It can at least report the installed packages, I guess?

I think there should be a popcon mode where it only reports the
installed packages. Reporting which programs was when last used down
to the second is something you can only activate on computers only
used by yourself or by a large enough number of persons. Once you
have a small number of other users on a machine, there is simply no
way to use it as it currently is.

        Bernhard R. Link


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100720161222.GA28187@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Holger Levsen-2
Hi,

On Dienstag, 20. Juli 2010, Bernhard R. Link wrote:
> I think there should be a popcon mode where it only reports the
> installed packages.

and it should be the default.


cheers,
        Holger

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Patrick Matthäi-3
Am 20.07.2010 23:18, schrieb Holger Levsen:
> Hi,
>
> On Dienstag, 20. Juli 2010, Bernhard R. Link wrote:
>> I think there should be a popcon mode where it only reports the
>> installed packages.
>
> and it should be the default.

Maybe the default value at d-i for popcon should be set to true (report
popcon statistics).
atm it is false.

Yes I know, that many people (I am including myself in general) think,
that we may abuse the data privacy with this default value, but..

1) if $user read the message, he still could decide to deactivate it
2) with `installed packages only' I think no privacy data will be submitted
3) we get more popcon data, which helps Debian and his users


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/4C461475.8080408@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Olivier Bonvalet-6
In reply to this post by Petter Reinholdtsen
Hi,

stats are based on the "HOSTID" contain in /etc/popularity-contest.conf ?

Some year ago all my virtual hosts have the same HOSTID because of the
way I was using to install them (rsync...).
Now my scripts take care of that, but maybe stats are erroneous on
server environment because of that "HOSTID", no ? Is there a way to
detect duplicates ?

Olivier

Le 20/07/2010 14:41, Petter Reinholdtsen a écrit :

> The number of submissions to the Debian popularity-contest collector
> is falling, and has done so for some time now.  This can be easily
> seen on<URL: http://popcon.debian.org/stat/sub-i386.png>.
>
> This is mostly caused by a fall in the number of Lenny installations,
> as can be seen from
> <URL: http://popcon.debian.org/stat/release-1year.png>.
>
> Anyone got any idea how to can get more machines to report to
> popcon.debian.org?  Or can there be some other problem causing the
> fall in the number of submissions?
>
> Happy hacking,
>    


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/4C4618B5.2090506@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

brian m. carlson-2
In reply to this post by Patrick Matthäi-3
On Tue, Jul 20, 2010 at 11:26:13PM +0200, Patrick Matthäi wrote:
> Maybe the default value at d-i for popcon should be set to true
> (report popcon statistics).
> atm it is false.

Absolutely not.  An installed Debian machine should be silent by
default.

> Yes I know, that many people (I am including myself in general)
> think, that we may abuse the data privacy with this default value,
> but..
>
> 1) if $user read the message, he still could decide to deactivate it
> 2) with `installed packages only' I think no privacy data will be submitted
> 3) we get more popcon data, which helps Debian and his users

I have in the past worked for a non-profit where no sort of release of
information was allowed.  If the program crashed, we could not make a
crash report because it might expose client information without us
knowing it.  This kind of policy is not at all unusual for non-profits,
especially those in the medical field.

Because of the potential for unexpected surprises, this will almost
certainly result in a lost of trust in Debian by users and
administrators.  I cannot emphasize how bad an idea enabling reporting
of even anonymized, non-personal data by default is.

--
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Christian Perrier
In reply to this post by Patrick Matthäi-3
Quoting Patrick Matthäi ([hidden email]):

> Maybe the default value at d-i for popcon should be set to true
> (report popcon statistics).
> atm it is false.

To people who would be tempted to followup on this and yell and "Thou
Shalt Not Do This" with arguments about ${privacy}, I suggest
thinking twice.

This has been discussed dozens of times since D-I automatically
installs popcon and *always* ruled out.

The question about popcon is asked at default install priority. Its
wording has been thought as carefully as possible, in order to explain
that popcon submissions are a helper for developers and not An Evil
Spy. This question is well translated to 62 languages.

There is nothing more we can do to have as many popcon submissions as
possible, really. If the number is decrasing, this is because the
number of Debian users who choose to install popcon is
decreasing. Very probably because the number of people who use Debian
is decreasing. Period.


signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Christian Perrier
In reply to this post by Petter Reinholdtsen
Quoting Petter Reinholdtsen ([hidden email]):
> The number of submissions to the Debian popularity-contest collector
> is falling, and has done so for some time now.  This can be easily
> seen on <URL: http://popcon.debian.org/stat/sub-i386.png >.

Well, would that be a tremendous scoop that the number of Debian users
is decreasing?

More particularly the number of users who choose Debian for their
desktop|laptop machine install?

It is quite obvious that most users who install dedicated servers,
virtualization machines, virtual machines inside the virtualization
machines, etc.... will not choose to activate popcon. And it is also
obvious that the people who are the most likely to activate it are
those who install Debian on a "personal" machine.

That number is decreasing. Is that *really* a surprise for anyone?
These days, in 2010, who is really seriously thinking that, apart from
a few hardcore geeks, someone who is considering to install a
Linux-based operating system on a personal machine will even include
Debian in his|her first choices?

So, our market are now server-style machines...or maybe still very
large installation of desktop systems based on Debian (Extremadura,
Munich, probably dozens of schools|universities...). And, sometimes, I
wonder whether those are also decreasing in some way (I'm always using
the same examples when I have to talk about these marge deployments).

We should probably face the fact that our growing slope is currently
negative and that will have consequences over some work we're
doing. Interesting discussion to have at DebConf, isn't it?



signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Asheesh Laroia
In reply to this post by Petter Reinholdtsen
On Tue, 20 Jul 2010, Petter Reinholdtsen wrote:

> Anyone got any idea how to can get more machines to report to
> popcon.debian.org?  Or can there be some other problem causing the fall
> in the number of submissions?

One thing Fedora does for their stats tools in the install is that if you
say "No" to stats collection, they ask you once more -- "But but it would
be so cool if you told us what hardware you have! Then we can figure out
what people use, and know what works! Pretty please?"

As I recall, it's for hardware stats, actually, not package installs.

CC:ing Mel Chua, who I *think* is the one who said that to me, so that she
can record as appropriate. (-:

So we don't have to change the default. We can nag more, as above.

We could also show a different message in the installer, one showing how
cool popcon is. Something like, "Did you know that Apache is the most
commonly installed web server in Debian, but nginx usage is increasing? We
know this because many people choose to report package installs through
popcon. If you say yes, we'll anonymously report which packages you have
installed and other statistics to Debian."

That's just my total rough-draft first pass, but there you have it. If you
like the idea we can refine it together (or you can ask me to give it a
better shot). (-:

-- Asheesh.

--
He was part of my dream, of course -- but then I was part of his dream too.
  -- Lewis Carroll


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/alpine.DEB.2.00.1007210150070.1180@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Andreas Tille-5
In reply to this post by Holger Levsen-2
On Tue, Jul 20, 2010 at 05:18:13PM -0400, Holger Levsen wrote:
> > I think there should be a popcon mode where it only reports the
> > installed packages.
>
> and it should be the default.

Why?  IMHO the number of packages which are *really* used is much more
interesting than all the packages which might be installed for dome
testing and just forgotten to purge.

And back to the topic: IMHO the relation between Debian users who are
using popcon and those who don't remains quite constant - at least I do
not see any reason why people should deinstall popcon if they previosely
decided to install it and there is no obvious sign why new installations
should get less acceptance for agreeing with popcon that this was
before.

If you ask me, the decreasing number of popcons is because people are
bored by a system with old versions of programs and are seeking for
alternatives and we will see a further decrease until Squeeze will be
released.  So probably the best idea to increase popcon numbers is to
fix some RC bugs to get Squeeze in shape.

Kind regards

        Andreas.

--
http://fam-tille.de


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100721071259.GA22158@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Giacomo Catenazzi
In reply to this post by Petter Reinholdtsen
On 20.07.2010 17:26, Petter Reinholdtsen wrote:
> The popularity-contest package also work fine with relatime.
>
> Thanks for bringing this up.  I guess we should look at the FAQ or
> something to make it clear on this point.  Where did you get the idea
> that popcon don't work with noatime?


I don't remember exactly. Possibly was a wrong extrapolation of
information in LWN: discussiong noatime/relatime it listed
few applications (popcon and few mail and backup programs) which
"required" atime functionality.

But I think I read also something in debian-devel or in an other
debian resource (years a go).

 > Is there some documentation we
 > could update to avoid others getting the same idea?


Personally a good post in planet about popcon (call of volunteers,
privacy and noatime) would help. But to reach sysadmin I think
we must create a wiki page, links it to the planet article and
let waiting the spreading of the "news".

ciao
        cate


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/4C46A30E.2070700@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Goswin von Brederlow-2
In reply to this post by Petter Reinholdtsen
Petter Reinholdtsen <[hidden email]> writes:

> [Jesús M. Navarro]
>> 1) Current reduction from April onwards is statisticallly
>> non-significant with a main trend steadly growing since Aug, 2007.
>
> Could be.  Hard to predict the future. :)
>
> I suspect it is a significant reduction, but I do not know the cause.
> Perhaps Debian have a reduced user base, people moving to other
> distributions.  Perhaps the popcon.debian.org server is overloaded and
> unable to handle more requests (unlikely, only ~14k submissions via
> http every day.  Perhaps some strange rumor have spread that popcon do
> not work on laptops, and more and more laptops are being deployed in
> the Debian community.  Or perhaps something completely different.  I
> do not know, but hope that bringing it up here might get more eyes to
> look at the issue and perhaps find a way to get more popcon
> submissions. :)

Does Ubuntus popcon report to popcon.d.o?

MfG
        Goswin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/87eiex6yls.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: The number of popcon.debian.org-submissions is falling

Paul Wise via nm
On Wed, Jul 21, 2010 at 4:14 PM, Goswin von Brederlow <[hidden email]> wrote:

> Does Ubuntus popcon report to popcon.d.o?

No, just to popcon.u.c:

http://patches.ubuntu.com/p/popularity-contest/popularity-contest_1.48ubuntu1.patch
http://popcon.ubuntu.com/

They also currently have almost 20 times as many popcon submissions as
Debian and continuing growth:

http://popcon.ubuntu.com/
http://popcon.ubuntu.com/stat/sub-i386.png

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/AANLkTinzHKVtgzVnJMJZD4OefS0L7YJzL_8kjpwkzLGW@...

1234 ... 16