16 ms·
Call of Duty: Warzone Caldera Data Set for Academic Use
- jchw 2y agoYet again the word "open source" is being used in a way that doesn't make any sense. We're going to wind up with a weird situation where "open source" means "free and open source" for software specifically but just means "available free-of-charge" for data and ML model weights. Which is strange. The word "free" is right there. This is not "source code", and it certainly isn't "open source" even if it was. I know this is a tangent, but unfortunately it bears repeating.
- jncfhnb 2y agoML weights are code
- jchw 2y agoI'll accept that if you would like. However, they are not source code if so. They are object code. And open source is about source code, not object code. (And this particular press release isn't about ML weights anyways, at least unless I'm grossly misunderstanding; it is just a dataset. So even failing this, it still doesn't really make any sense.)
- jncfhnb 2y agoNo it is not object code unless you want to get so stupidly pedantic that you want to argue a Python script in a zip file can’t be considered open source because it’s compressed. The model pickles unpack back to their original form. The picked binary forms are merely for convenience.
- jchw 2y agoLook, please go do research as to what "object code" and "source code" are before saying my argument is "stupidly pedantic". I'm not elaborating because the example you gave has nothing to do with what I said.
- jncfhnb 2y agoYour analogy does not make sense. ML weights are distributed in binary form, like object code, but it is nothing like compiled binary. It’s just temporarily in binary form for convenience. It unpacks directly into its original form. This is not a technicality like “technically up can reverse engineer or modify binary code”. The binary form of model weights is just a fancy zip file format that is useful because they are so large that text is impractical.
- jchw 2y agoSource code is human readable. Object code is not, and produced from some mechanical process. Model weights are not written by hand. You don't manually tweak individual weights. You have to run a training process that has multiple "raw" inputs. Trying to read model weights directly is no better than trying to read object code directly. Heck, reading object code directly is probably easier, because at least it's just machine code at the bottom; I will never be able to comprehend what's going on in an ML model just by reading the weights. The closest thing to "source code" in ML models would be the inputs to the training process, because that's the "source" of the model weights that pops out the other end. If the analogy doesn't make sense, that's because ML models are probably not really code in the same sense that source code and object code. (It may be tempting to look at "ML weights" as source code because of the existence of "closed-weight" API services. Please consider the following: If Amazon offers me a unique database service that I can only use with Amazon Web Services, and then releases a closed-source binary that you can run locally, that is still closed-source, because you don't have the source code.)
- jncfhnb 2y ago“Human readable” is not a requirement. Visual programming code breaks down to some obtuse data structure. But with the right tools, it’s easy for humans to interact with it. Visual programming node workflows can be open sourced. ML models are the same. Tooling is required to interact with it. The limits of your human understanding do not determine if something is open source. Otherwise a really complicated traditional program might be argued as not open source. You can individually explore specific vectors and layers of a model and their significance. Produced by a non mechanistic process is not a requirement. I can generate a hello world script with code, and open source the hello world script. It does not matter how it was formed. I do not need to open source the hello world generator either. Data and training code is not source code of the model. That is the source code of a model maker. That’s `make_hello_world.py` not `hello_world.py` The closed source database is not a correct analogy. Excluding unreasonably difficult efforts to decompile the binary, you CANNOT modify the program without expecting it to break. With an ML model, the weights are the PREFERRED method of modifying the program. You do NOT want the original data and training code. That will just be a huge expense to get you what you already have. If you want the model to be different, you take the model weights and change them. Not recreate them differently from scratch. Which is the same for all traditional code. Open source does not mean I provide you with the design documents and testing feedback to demonstrate how the code base got created. It means you get the code base. Recreating the codebase is not something we think about because it doesn’t make sense because we have the code and we have the models.
- shevis 2y agoNo, they really aren’t and I’m not sure why I keep seeing this take. ML weights are binary and it’s painfully obvious. They are the end result of a compilation process in which the training data and model code are compiled into the resulting weights. If you can’t even theoretically recreate the weights on your own hardware it isn’t open source.
- jncfhnb 2y agoML weights are not binary. They are modifiable. If produce a program that outputs a hello world file, I can open source the hello world script without open sourcing the hello world generator.
- deleted 2y ago[deleted]
- kennyadam 2y agoTechnically, but it feels like you're intentionally missing the point being made. Sure, providing the weights is very useful given the cost of generating them, but you can't exactly learn much by looking through the 'code', make changes and gain an in-depth understanding in the same way you can from the code provided by an actual open source project.
- jncfhnb 2y agoYou absolutely can and people do all the time. There are mountains of forks and dissections and improvements on open source models.
- i_read_news 2y agoWe can also say binaries are code, but if we are being pedantic that likely isn’t the source code that generated the binary (I also doubt the intention of hand writing binary or manually inputting billions of weights). I’d reckon that’s why it’s called open source, not open code or open binary, as the source code that generates the data is distributed. I’d actually just call this for what it is - open weights.
- jncfhnb 2y agoBinary is not the equivalent of models. Source code is the equivalent of models. It doesn’t matter if a machine generated source code or a human did for it to be open source code.
- EnigmaFlare 2y agoYou keep asserting this but without any reason. Do you have a reason? It seems to go against the general open source idea of source code being convenient for people to modify.
- jncfhnb 2y agoML weights ARE convenient for people to modify. You can go look at the dozens of modifications of diffusion models being produced, daily, on civit ai. It’s very easy.
- EnigmaFlare 2y agoWould you say that once a model is trained, there's no need to go back and re-train it, even if you want to, say, remove some material from the training set? Anything can be done just with the weights? That's a big surprise to me. Of course people hack binaries too, and binaries are obviously not source code. I once edited a book in PDF form because we didn't have the original Word/whatever document. It's not hard but a PDF still isn't considered to be source code for documentation despite that.
- fxd123 2y agoThe repo contains some source code, so therefore it's open source
- ssss11 2y agoFeels like a covert way to destroy the term “open source” by making it meaningless over time.
- WheatMillington 2y agoYes it's all one big conspiracy.
- 01HNNWZ0MV43FF 2y agoNo, it's a Schelling point but evil
- highcountess 2y agoI find it rather odd that after all the years of exposed and revealed conspiracies too numerous and pervasive to even necessitate listing any of them, people like you just reject the notion that any additional, unknown conspiracies may exist. It is an odd phenomenon among humans that I at least don’t quite understand, the seeming tendency to ignore or dismiss possibilities of proven negative outcomes … for whatever reason. “I know all those other conspiracies I dismissed all turned out to be true, but I am sure I would know if there were any additional conspiracies” … totally ignoring one’s track record. It appears to be the same kind of mentality of “hey, you know who we should trust with our lives … the government made up of people who lie to us, steal from us, and mass murder on a regular basis; that’s who we should give control over to.” People conspire, I’ve witnessed it personally numerous times; sometimes for greedy business reasons, at other times to mass murder and commit genocide on a scale not seen since. Humans conspire, even if sometimes only because they’re not prevented from doing so naturally.
- ssss11 2y agoIt’s probably convenient for them to dismiss this as it aligns with whatever goals they have..
- aaron695 2y ago[dead]
- deleted 2y ago[deleted]
- hgs3 2y agoRight or wrong licensing source code separately from data isn't a new thing. I can think of some very famous video games that have released their source code under a Free Software license, but kept the game data proprietary. According to the FSF there is a separation between data and code [1] (search for "data" on that page). They specifically say that data inputted or outputted by a program isn't affected by the programs license which indicates a separation from their perspective. [1] https://www.gnu.org/licenses/gpl-faq.en.html https://www.gnu.org/licenses/gpl-faq.en.html
- userbinator 2y agoAccording to the FSF there is a separation between data and code Which of course is a complete denial of the reality. Code is data, and data is code. That duality is the crucial reason why general-purpose computers are so powerful. The only ones to profit from trying to make a distinction, as usual, are the lawyers and corporations behind them who seek to restrict instead of empower. Especially in this era when decompilers are close to "perfect" (and can sometimes even be better than reading the original source code!), and with the rise of AI, IMHO the whole idea of "source code" being somehow more special than the executable binary is quickly losing relevance.
- nolist_policy 2y agodecompilers are close to "perfect" (and can sometimes even be better than reading the original source code!) Citation needed.
- mplewis9z 2y agoI have definitely read some teammates’ code that felt like it would be more readable doing a compiler-decompiler round-trip. Never actually did it, but I doubt it would be less readable than that seemingly intentionally obfuscated garbage.
- InDubioProRubio 2y ago
- dkersten 2y agoThe article claims it’s open source (which it clearly isn’t, especially since they say things like “open source for non-commercial use” which is a bit of a contradiction), but the GitHub makes no such claim only stating that the OpenUSD format is open source.
- bee_rider 2y agoI’m confused as to why the convention isn’t to consider ML weights data-sets instead of any type of code (closed or open).
- jncfhnb 2y agoModel weights are functions. In the same way `lambda x: x > 0.25` is a function.
- blitzar 2y agoThe phrase "open source" is itself open source and is freely available for use, modification and redistribution.
- insomniacity 2y agoNot exactly - https://opensource.stackexchange.com/a/8369 https://opensource.stackexchange.com/a/8369
- fragmede 2y agoOpen Source, with the capitals, however, is not, and is a trademark of the Open Source Initiative (OSI). https://opensource.org/trademark-guidelines https://opensource.org/trademark-guidelines
- insane_dreamer 2y agoNo, it's not. From the page you linked to: > OSI, Open Source Initiative, and OSI logo (“OSI Logo”), either separately or in combination, are hereinafter referred to as “OSI Trademarks” and are trademarks of the Open Source Initiative.
- fragmede 2y ago> In all cases, use is permitted only provided that: > the use of the term “Open Source” is used solely in reference to software distributed under OSI Approved Licenses.
- captainhorst 2y agoThe map data is provided in the USD format which is a 3D authoring and interchange format that can be used with a lot of software. Unlike the final optimized data used by the game this doesn't require revere engineering and can be seen as source data that is in fact useful for graphics researchers and game developers.
- bogwog 2y agoThese files are source assets, which is as close to source code as you can get with non-code stuff. For regular people who didn't drink the OSI koolaid, this is a perfectly valid and logical use of the term "open source". I don't know if that's the angle you're coming from, or if you just didn't know what usd was, but either way this is a good release.
- blopker 2y agoLink to the data is in a Github repo at the bottom: https://github.com/Activision/caldera https://github.com/Activision/caldera Reading the article, they don't seem to know what people should do with it. It feels like a recruiting tool more than anything, especially given the non-commercial license.
- SkyPuncher 2y agoAs an avid CoD player, I literally have no idea why this would be useful. Map data isn’t really interesting. The player data seems far too low of resolution to be meaningful.
- a_e_k 2y agoThese sorts of data sets can be useful for graphics research, particular as a data set to test ray tracing algorithms on. See for example, the Moana Island data set. [1] I definitely foresee papers on BVH construction using this scene. For graphics research in academia, there's a dearth of real-world data sets like this, so the ones that do get released are gold. And for graphics research in industry, one may have access to good internal data sets for development and testing, but getting permission to publish anything with them tends to be a giant hassle. It's often easier to just use publicly available data sets. Plus, that makes it easier to compare results across papers. [1] https://www.disneyanimation.com/resources/moana-island-scene/ https://www.disneyanimation.com/resources/moana-island-scene...
- Takennickname 2y ago"Find us an AI use case that we can then turn around and market without compensating you for it you researching piece of shit. Sincerely, Activision"
- rasz 2y agoPlayer movement data can be used to build aimbot with undetectable lifelike movement. Thanks Activision!
- reportgunner 2y agoThis is not even necessary since current cheaters seemingly can't be detected anyway.
- bogwog 2y agoI wonder if the data includes information about which players were banned for cheating? That could open the door to new research into cheat detection.
- bee_rider 2y agoEnemy AI has been pretty stagnant for the last couple decades, right? Can somebody use this to make less inhuman bots?
- chrishare 2y agoI wonder what that would look like for the twitch shooter genre?
- IvanAchlaqullah 2y ago> less inhuman bots In games, player don't want AI that 100% strong (it's not fun), what we want is AI that make mistakes like human do. So it's possible (assuming if the datasets is good), in fact it's already done in chess[1]. > Maia’s goal is to play the human move — not necessarily the best move. As a result, Maia has a more human-like style than previous engines, matching moves played by human players in online games over 50% of the time. Also something that I just realized: in this particular case, we want the AI to be biased like human, which is easier to do since the bias is already in the datasets. AI safety is the exact opposite, which is harder if not impossible. [1] https://maiachess.com/ https://maiachess.com/
- jncfhnb 2y agoWe don’t even want that. Human like AI in a game like this would be really annoying for that vast majority of players that absolutely suck. People who want to play against AI generally want something that’s really dumb.
- cstejerean 2y agoPlaying against an AI that's really dumb gets boring quickly. Playing against an AI that's way too good gets annoying quickly. I want an AI that can play like a human at my level would, such that the game is competitive and fun.
- jncfhnb 2y ago
- Andreas1303 2y ago[flagged]
- mileseva 2y ago[flagged]
- deleted 2y ago[deleted]
- 2OEH8eoCRo0 2y agoVery cool. Does anyone remember how Bungie would release heatmaps and other data for each Halo match?
- Gracana 2y agoAbsolutely. I used to love all the data you'd get after an ODST match. I went digging for the online stats a while back and came up empty-handed... sad to see it all seems to be gone now.
- dagmx 2y agoA lot of the comments here are very cynical, perhaps because they’re focused on the license or the use for gaming. However as someone in the graphics community, these kinds of assets are great for researchers and demo purposes. Other scenes like this are the Disney Moana Island, Intels Moore Lane house, Sponza, various NVIDIA scenes, Amazons bistro, and animal logics Alab2 scene. Khronos also maintains a set of test assets for the same purpose with glTF. When we develop content creation applications, they’re great for benchmarking ingest and making sure we have good feature coverage. They’re great for graphics researchers to have shared bases for data processing, rendering and other important R&D. The non-commercial aspect just means you can’t use them for commercial marketing, but they’re hugely beneficial for any kind of graphics research. Having real production quality data is a huge undertaking for researchers to do in addition to their own novel work. Thus far, many sample assets have been simple standalone assets, film quality production assets, or archvis. Activision releasing something from a AAA game is a huge boon for people targeting that market. I’ll also call attention to Natalya being involved. She’s recently joined Activision as CTO , but has been a very influential graphics engineer with a long and storied career before that. She has long helped run the excellent Advances in Real-time Rendering courses at SIGGRAPH (https://advances.realtimerendering.com/ https://advances.realtimerendering.com/) and I believe this release comes from the same intention of mutually advancing shared knowledge.
- deleted 2y ago[deleted]
- ghawr 2y agoAre there similar data sets for video & audio assets?
- aubanel 2y agoGreat, now I can finally find the top 5 hiding places for these friggin campers!