GZStats: A quick rundown

[Rachael]

One thing both myself and Graf have been working on is a stats collector. We wanted to make sure we had a working implementation before unveiling it.

What is it?

This is a little mini background function that runs at the start of GZDoom which will send us some very generalized anonymous statistics about the hardware it is being run on.

Why are you doing it?

The purpose of this is to allow us to make informed decisions about hardware deprecation - i.e. if it is time to move on from OpenGL 2.x, whether to drop support for 32-bit processors, how viable a Vulkan engine will be at this point, etc. This is to avoid blindly making decisions about deprecating hardware or making assumptions about it without knowing how many people it will actually affect.

How often does it run?

Depends on how much you wipe your config. If you tend to hold on to your config like most people do, only once per renderer. So it will run the first time you run in Software mode, and the first time you run in OpenGL.

I don't have internet. Will GZDoom still run?

Yes. This little process was very specifically designed to be non-intrusive and, if it cannot find the server it will save its statistics info for next time. It only attempts one connection before it gives up, and GZDoom will run in the foreground while this is happening. The whole thing has absolutely no interaction from the user and it should not even cause the game to stutter. It's also a quick-fire - i.e. attempt it, and it's either successful or not, but either way it's done.

But my privacy!

This was very specifically designed in such a way that there is absolutely no way we can personally identify you. At worst, you'll have an entry in our web server's access log. Big whoop - guess what, you leave plenty of those entries no matter where you go on the web! Anyway, outside of very non-identifiable statistics about your hardware, no personally identifying information is ever collected or sent. We won't even know what wads you're running. We won't even know if you used Doom or Doom 2. Or Heretic. Or Strife. It doesn't even tell us that. All we'll know is there's yet one more person who has a dual-core processor and still running OpenGL 2.something on Windows 10 32-bit, we can't even tell if it's 2.0 or 2.1. We won't even know how much RAM you have.

What info does it send?

To be very specific, it only sends the following information:

Operating System Type: (GZDoom picks only one of the following)
XP (or older)
Vista/7/8/8.1 32-bit
Vista/7/8/8.1 64-bit
Win10 32-bit
Win10 64-bit
A special case if you're using WOW64 (you downloaded 32-bit but you're using a 64-bit OS)
Mac 32-bit
Mac 64-bit
Linux/BSD 32-bit
Linux/BSD 64-bit
Linux/BSD ARM
Linux/BSD PPC

Processor Cores: (GZDoom only picks one of these)
Single, Dual, Quad, Hexa, or Octa (or higher)

Renderer Type: (GZDoom only picks one of these)
Software (No HW2D)
Software (Legacy Direct3D, SM 1.4)
Software (Modern Direct3D, SM 2.0 or later)
Software (OpenGL)
OpenGL (Legacy <3.3)
OpenGL (Modern >=3.3)
Using OpenGL (but has Vulkan Support)

[TerminusEst13]

Can I turn it off?

[wildweasel]

Can I turn it off?

As much as disabling this would defeat the point of such data collection, I did some poking around - it can be effectively disabled by changing one value in the INI file:

Code: Select all

[GlobalSettings]
sys_statshost=gzstats.drdteam.org

Just blank that setting (i.e. set it to sys_statshost= with nothing after it) and it won't do anything. Though, as mentioned, not only does this massively defeat the point of such data, there's barely any collected in the first place.

Source: d_stats.cpp

[Hellser]

I think this should be opt out. Even Minecraft, now owned by Microsoft, has an option to opt out.

[Kinsie]

I believe Dolphin Emulator, which has a similar system, asks the user to opt-in via a no-frills generic OS dialogue box on first startup. Probably wouldn't be enormously difficult to add in, even at the 11th hour, and it'd save a bit of pointless drama later.

[Graf Zahl]

Surely such a thing can be added and most likely will be before doing a release.

But you are aware that switching this off would eliminate your system from the pool of data, right?

So, imagine two scenarios:

1. You own an old system and refuse to participate. It will mean that those low end systems will be underrepresented and we then decide to remove support because apparently nobody uses them.
2. You use a new system and refuse to participate. This will mean that those old low end systems will be overrepresented and we decide that keeping support is more important than optimization.

In both cases you will only hurt yourself by not doing it.

[Hellser]

Showing a screen of what exactly will be sent will probably help ease the minds of those who are weary of opting in or out (again, Minecraft does this). People will think that GZDoom is becoming an information-collecting program rather than a game. Especially in todays day and age where privacy is a concern.

[Kinsie]

Surely such a thing can be added and most likely will be before doing a release.

But you are aware that switching this off would eliminate your system from the pool of data, right?

So, imagine two scenarios:

1. You own an old system and refuse to participate. It will mean that those low end systems will be underrepresented and we then decide to remove support because apparently nobody uses them.
2. You use a new system and refuse to participate. This will mean that those old low end systems will be overrepresented and we decide that keeping support is more important than optimization.

In both cases you will only hurt yourself by not doing it.

I'm aware and I agree, but making things opt-in is a show of good faith and (more importantly) likely avoids a shitfight. I think we can agree that we've had enough shitfights over pointless things in the past without adding to the list.

[Rachael]

Showing a screen of what exactly will be sent will probably help ease the minds of those who are weary of opting in or out (again, Minecraft does this). People will think that GZDoom is becoming an information-collecting program rather than a game. Especially in todays day and age where privacy is a concern.

This is exactly what Graf has access to:


This is exactly what GZDoom sends to the server:


I have access to the logs but I won't view them on this particular domain, nor am I interested. But for those of you who are truly paranoid, yes your IP address appears in the log, yes I, MancII and his staff, and Dreamhost has access to it (*no one else* has access to it), and as ZZYZX pointed out your IP address can be linked to your forum account. (But honestly, how can I use that information to harm you, even if I wanted to?)

[Sgt. Shivers]

If this collects our IP addresses and our IP addresses can be linked to our forum accounts then doesn't that make this kind of non-anonymous?

[dpJudas]

If the thing was coded to connect those two dots, then yes. But it isn't, so no, we don't know who got which hardware. In fact: we don't even care! We just want to get a rough idea of what users have - like our own little Steam hardware survey for future decisions. That's all there is to it.

[Kotti]

If this collects our IP addresses and our IP addresses can be linked to our forum accounts then doesn't that make this kind of non-anonymous?

It's at least a lot more anonymous than clicking some Facebook-"like" button on some random website. If you are this paranoid about anonymity when posting in some forum, do what I normally do and use a VPN service that gives you an IP that cannot be connected to your real location. Use a different VPN for different parts of your life if you are really paranoid and feel like it might serve you. Here's a link to one of the services I use: https://www.tunnelbear.com/

But ultimately, if you are concerned about privacy, best sever your internet connection for good. 99.9% of the internet collect far more data than what that tiny bit of info that gets sent here contains. Each website you visit with your browser already knows more about you and your habits, thanks to all the metadata that gets sent along with your request.

So what if Rachael can see if you participated in this survey? What does that tell her that she cannot get from just reading your post history? Let's see what I already know about you from publicly visible data on this forum. Here's what I could get, with the revealing parts crossed out to protect you. Keep in mind that all the hidden stuff is publicly visible to any forum user and partially even retrievable through advanced Google search. I know precisely what fills the gaps by looking around your posting history for maybe two minutes.
You have been posting here for roughly ??? years.
Your main interest is the ??? thread which almost makes up ???% of your entire posting on this forum.
You once stated "I prefer the ??? renderer".
And back in ??? you were using Windows ???.

Someone interested in you could get to most of that data even without logging in by using Google's advanced search options and by investing a bit more time would get to know you even better. Now, what else can someone add to all that knowledge by knowing a very condensed summary about your current hardware?
I'd say nothing.

I hope this puts the entire thing into relation with reality. Personally, I think that surveys like this will ultimately help improve software, this cannot be compared to Microsoft's clandestine use of telemetry in any way.

[Marrub]

I like this, I also don't like applications randomly making web connections. And if it doesn't follow global proxy settings, even accidentally, it is potentially a security risk to those who just want to play a game. Making this opt-in with a simple "Can we send simple, anonymous hardware statistics on occasion to the developers?" would not turn away any users. It doesn't skew results, either – that's like saying people not voting in a poll are causing skewed results. If they didn't know the poll existed or didn't want to vote, they aren't really skewing anything, are they?

[Kinsie]

I have access to the logs but I won't view them on this particular domain, nor am I interested. But for those of you who are truly paranoid, yes your IP address appears in the log, yes I, MancII and his staff, and Dreamhost has access to it (*no one else* has access to it), and as ZZYZX pointed out your IP address can be linked to your forum account. (But honestly, how can I use that information to harm you, even if I wanted to?)

While I know what you're talking about and getting at (since it's a generic HTTP request much like the many anyone reading this has made loading this page), wording it like this is kinda unnecessarily -inducing. And I feel that's where this is primarily being fumbled - the messaging. Luckily, it hasn't properly shipped yet, and making it opt-in would assuage most of the concerns.

[Graf Zahl]

True, but let's not forget that we haven't even discussed yet, how to convey the message to the user. The main concern, obviously, is that we do not want to scare away those non-forum visitors which constitute the silent minority. As developers we absolutely NEED the info to decide how to continue with legacy support so this needs to be done in a way that people feel they can participate rather than being spied upon. Most of this requires very careful wording.

Oh, and let me add to Kotti's post. Every website you visit, especially when Javascript is active, gets magnitudes more info about you, your computer and your habits just by going there once. Technology has become good and fast enough, that they can do a unique fingerprint of every computer that visits their site, if they are so inclined, without you even noticing that they do. I've also come into contact with some truly pervasive and hostile JS code at work that is dead set on circumventing any security measures a browser can use to reliably reidentify you if you revisit a site from another IP or with cookies deleted. I have long come to the conclusion that if you need a minimum of privacy on the internet the least you need is the NoScript plugin and for visiting sites you do not really trust, use the Tor browser - and don't use smartphone apps! Nothing leaks more private data than those convenient gadgets, and they give you no control at all over what gets sent!!! If that is too inconvenient, don't be surprised if people really get to know everything about you in no time.

But at the same time people get angry at obviously harmless information gathering because it is honest and tells up front that it gets done, while they should be far more concerned about the hidden data gathering that happens behind their back. What a weird world... Well, as they say, ignorance is bliss...