Talk:WebAPI/Feedback

Suggestions

Things that should be fixed in the Steam Web API system

• JSON and VDF should use arrays rather than objects containing arrays. Having them structured differently from XML complicates code unnecessarily (and result.items.item[1] is redundant compared to results.items[1]). This seems to have been designed to preserve the element name used for the individual objects in XML (allowing multiple arrays to be defined in the event of mixed types, which should be structured as multiple arrays in XML as well). When everything in an array is of the same class, nobody in JSON cares about that class's name (only its index in the containing array, and anybody using VDF is using code that was written for the original data file, which is structured this way.

Short version: this

 "result": {
   "status": 1,
   "items": {
     "item": [
       {
         "id": 18446744073709019323,
         "defindex": 125,
         "level": 1,
         "quality": 3,
         "inventory": 2149580806,
         "quantity": 1
       },
       {
         "id": 1029705,
         "defindex": 36,
         "level": 5,
         "quality": 3,
         "inventory": 2147483655,
         "quantity": 1
       },
     ]
   }
 }

should look like this:

 "result": {
   "status": 1,
   "items": [
     {
       "id": 18446744073709019323,
       "defindex": 125,
       "level": 1,
       "quality": 3,
       "inventory": 2149580806,
       "quantity": 1
     },
     {
       "id": 1029705,
       "defindex": 36,
       "level": 5,
       "quality": 3,
       "inventory": 2147483655,
       "quantity": 1
     },
   ]
 }

--STUART 04:23, 5 July 2010 (UTC)

The lists are constructed the way they are because that's the way the underlying webapi system in Steam works. All lists in all Steam webapis are that way. That isn't likely to change because there are so many APIs involved. It's ugly, but it works. Hopefully it isn't causing any great pain.Joe 18:34, 23 November 2010 (UTC)

It isn't, and I figured that it had something to do with the layout of objects returned from the APIs this one uses. At worst it's a minor inconvenience to people if they're writing sane code. -- Lagg 02:39, 24 November 2010 (UTC)

Methods that should be added to ITFItems_440

• A method that returns all of a language's strings for a given language (like resource/tf_english.txt). This would allow simpler caching of GetSchema with regards to translation (clients would only need 1 GetSchema return for all languages, and could request each language individually without needing to get all of the schema data again, increasing the signal:noise ratio significantly for translation). --STUART 04:22, 5 July 2010 (UTC)
  • Would the ability to set "language=All" help? Joe 18:36, 23 November 2010 (UTC)
• A method that returns the latest revision information for each of these, so systems know when they need to make the heavy call to GetSchema and other static methods. --STUART 04:22, 5 July 2010 (UTC)
• A method that returns inventory images, or links to images. Currently the biggest hurdle of an update is digging all of the new data out of the tf2 gcfs. The three files at the moment are: items_game (partially covered by the GetSchema), tf_<lang> (Partially covered by GetSchema), and the backpack images (No solution). The images are also the most time consuming part. FireSlash 17:02, 5 July 2010 (UTC)

Biggest hurdle? Extract tf/materials/backpack and run it through VTFEdit's batch converter. Netshroud 23:27, 5 July 2010 (UTC)

As far as time consumption yes, since I also need tor resize and recompress images. items_game and tf_english are just a matter of extracting and uploading each file. Where in the images I have to sort through all the images in the gcf, dig out the new ones, extract convert resize and upload. If the images were available online, I could automate the process. FireSlash 00:58, 6 July 2010 (UTC)

Things that should be added to the next version of GetSchema

• The revision number, so systems can compare it (for stored data) against the latest revision (as described above). --STUART 04:22, 5 July 2010 (UTC)
• The defindex of attributes for items, rather than the attribute's name and class. If GetPlayerInventory uses defindex for its attributes, GetSchema should too. --STUART 04:22, 5 July 2010 (UTC)
• Add missing fields used_by_classes and craft_class FireSlash 16:17, 5 July 2010 (UTC)
• Add field propername to allow for simple construction of item name. i.e "A <Community> __" vs "The __", and "The Hound Dog" vs "Hound Dog". Ath 14:27, 12 August 2010 (UTC)
• Descriptions for the items would be very much appreciated. Netham45 18:39, 10 October 2010 (UTC)

Things that should change in future versions of GetPlayerInventory

• Ditch the "quantity" field. With a unique ID on the item, how can this be anything but 1? --STUART 04:22, 5 July 2010 (UTC)
• Separate the "inventory" field into an "inventory_slot" value and "equipped" object (with a "true" for each class the item is equipped for, and empty if no classes have the item equipped). This is a Web API: consumers shouldn't have to do binary operations to parse it. --STUART 04:22, 5 July 2010 (UTC)
• Add a "timestamp" field, similar to the GetGoldenWrenches method. This would be useful for statistics and curiosity of users. --AltPluzF4 11:44, 10 July 2010 (UTC)

Alternative WebAPI profile query methods

• Allow developers to pass a vanity URL "ID" as an argument to the WebAPI or more easily convert a vanity URL to a SteamID64. The Steam Community XML feed breaks easily and does not properly sanitise community-side user input. (I.e. Non-UTF8 characters present in feed) Ath 23:18, 10 August 2010 (UTC)

One thing you could do until this happens is just search for the tag without passing it to a full parser. It's less lines of code and works regardless of encoding on the rest of the page. This is what I do in steamodd, and this way you only have to rely on the tag name not changing. Lagg 10:35, 11 August 2010 (UTC)

I make use of other parts of the XML too (May as well if I have to fetch it anyway :( ), so passing it to PHP's SimpleXML lib (through utf8_encode first) is simple and effective enough I find. Thanks for the suggestion though. Ath 14:27, 11 August 2010 (UTC)

Another possible solution that'd remove the need for the community XML, would be to add an additional ?sid64=1 argument to community profiles, which would just returns the profile's SteamID64. This would be very useful if also applied to the community's Search page; But instead returning a comma-delimited list of the top-x results. This would greatly simplify profile discovery, and allow the XML to be deprecated in favour of GetPlayerSummaries. Ath 16:58, 25 November 2010 (UTC)

Backpack size discovery

• With the introduction of backpack extenders the size of an individual's backpack is not certain; An attribute declaring the total number of pages would be helpful in simply returning only those pages that are available to the end-user. ~ Ath (talk) 15:58, 19 December 2010 (UTC)

We know that they extend it by 100 cells, this is enough to pad if the position of an item is greater than the default size you set. -- Lagg 16:00, 19 December 2010 (UTC)

The most logical way to do this is find the item with the highest position in the persons backpack. Then you know how many pages there are, with the added bonus of not having to show all 10 pages if the user isn't using the last few. :) --Pc Madness 16:15, 19 December 2010 (UTC)

For accuracy I would prefer to display the number of pages that actually exist vs those that are used; This both reflects the behaviour of the actual backpack, and provides an end-user with an overview of how much room they/the owner actually has. ~ Ath (talk) 16:40, 19 December 2010 (UTC)

We can "guess" how many pages the player has, but it would be much better if the API provide such data. --TheJCS 01:59, 21 December 2010 (UTC)

Inline gift metadata

• To my current understanding, date and the gift giver's SteamID64 are encoded into attributes 185 and 186 on gifted items; requiring a secondary API query to retrieve the gifter's name. Could this information be provided inline? ~ Ath (talk) 04:01, 25 January 2011 (UTC)

Problems

Breaking GetPlayerItems Bug

With certain backpacks (for example, 76561197967454254, one of the PCGamer employees) there appears to be a floating point error inserted into attribute values without being checked before hand. This results in invalid JSON that most parsers will fail on.

Example:

							{
								"defindex": 127,
								"value": 4294967294,
								"float_value": -1.#QNAN0
							},

-- Lagg 09:43, 10 November 2010 (UTC)

Oh joy. Glad to see serious issues like negative IDs were fixed though! Ath 19:42, 10 November 2010 (UTC)

The next update fixes this by not providing a float_value for values that are invalid. Long term I intend to provide more useful output for attributes than their raw value, especially in the case of particle effects like this one. Joe 23:33, 17 November 2010 (UTC)

Invalid Encoding in Certain Item Names, Descriptions, Personas, and Real Names

It appears that custom item names, descriptions, and community profile names and personas (and probably anything else that takes user input) isn't being sanitized before the JSON is generated. Invalid character encoding is also invalid JSON. -- Lagg 19:50, 10 November 2010 (UTC)

If you see an example of an account with a custom item name or description that's broken in this way, please post it here. I believe that there's a bug in our encoding of names somewhere, I just need a repro case. Joe 23:31, 17 November 2010 (UTC)

Joe, would it be possible for you to grant either myself or Lagg a name/desctag to attempt to repro this? I had a report of an effected profile, but it's no longer exhibiting any issues. Thanks. Ath 23:35, 17 November 2010 (UTC)

Hmm, is there anything that can be done about the characters being used by this guys backpack? 'TomasMarijanovic' It results in invalid JSON, even though it appears to be valid UTF8. He's got a backwards character in there as well which has some nasty side effects: http://www.tf2items.com/id/TomasMarijanovic hover over the Headtaker on page 2. --Pc Madness 05:21, 25 November 2010 (UTC)

Fantastic, I've been looking for a live example for a while now. It looks like an invalid control character in his headtaker's description. I don't think I'm going to work around this one. -- Lagg 13:50, 25 November 2010 (UTC)

Ah ha! I'll cull control characters from the names and descriptions. Thanks for the example. -- Joe 19:38, 26 November 2010 (UTC)

The fix for this is live now. Please let me know if you see any other examples of invalid encoding on names. -- Joe 00:22, 3 December 2010 (UTC)

Think I've got an example of more invalid characters in JSON. TF2B isn't getting any output from json_decode() @ http://tf2b.com/?id=yeszz&debug=1 and Community profile @ http://steamcommunity.com/id/yeszz. According to the account owner the description for his Sticky Launcher "glitched out" and was replaced by various symbols. ~~ Ath 00:17, 4 December 2010 (UTC)

It looks like that name is parsing fine in some JSON parsers. Are you getting any errors or exceptions from json_decode()? (If the account owner files a support ticket about the glitched name he can get it fixed.) -- Joe 17:56, 21 December 2010 (UTC)

I'm seeing an error in optf2's logs as well. There's an invalid unicode character in it. What parser are you using, Joe? It should fail proper on invalid encodings. It appears that the problematic description is on 165438789. -- Lagg 20:45, 21 December 2010 (UTC)

json_decode() uses a strict parser as far as I know so it'll be choking on the invalid Unicode character as Lagg stated. Is it okay to pass the Steam support recommendation back to the owner? Or would you prefer to examine the issue closer first? ~ Ath (talk) 22:44, 21 December 2010 (UTC)

List of Problematic Attributes

Please feel free to edit or correct this list if I missed anything (I want to keep it all here to be more visible to Joe) -- Lagg 03:57, 14 November 2010 (UTC)

Attributes with no description string

27 alt-fire disabled

29 alt-fire is vampire

43 scattergun no reload single

53 fire retardent

116 mod soldier buff type

117 dmg falloff increased

122 mod demo buff type

167 disguise damage reduction

183 aiming movespeed decreased

184 duel loser account id

189 elevate quality

Attributes with undefined description_string

192 referenced item id low

193 referenced item id high

Attributes with incorrect description strings

195 always tradable

196 noise maker

144 lunchbox adds minicrits

Attributes attached to items they shouldn't be attached to

144 lunchbox adds minicrits (attached to the GRU)

The behavior on referenced_item_id_* is intentional. These are hidden attributes, so they don't have a description. Some of the others that have the wrong descriptions are also hidden and need their descriptions removed. The next webAPI update (which will happen soon) adds the hidden flag to what is provided for attributes, so it's easier to detect these attributes. I'll look at the others. -- Joe 22:20, 15 November 2010 (UTC)

Great, but I strongly suggest setting description_string to null or an empty string on those attributes. Otherwise I need to mark it as optional in the docs. -- Lagg 01:46, 16 November 2010 (UTC)

These are fixed as of the next update. Hidden attributes now never return a description_string or description_format field. I'll update the docs to reflect that when the update goes live. There are still a couple of unused attributes (alt-fire disabled and alt-fire is vampire, for instance) that don't have strings. If we ever use them for anything they will get strings at that point. Joe 23:29, 17 November 2010 (UTC)

Alright, most of these appear to be fixed but some of the hidden/unused ones still have the description token there instead of being empty. It doesn't matter at this point though since we have the hidden flag now (Many thanks, Joe). However the GRU still have the minicrit attribute attached, not sure if this is intentional (See here). -- Lagg 02:53, 20 November 2010 (UTC)

It's intentional. The name is poor, but since the attributes are hidden and the name never shows up anywhere but the webapi it hasn't been a big priority to change. Joe 02:54, 20 November 2010 (UTC)

Versioning

What's the point of having the potential for versioning (e.g. GetSchema/v0001) if it's not used? The recent update (November 20) removed description_format and desciprion_string from a whole handful of attributes, in turn breaking my iOS application. I was saving data out to Core Data and had the description format and description string as required attributes for the backpack item attribute. Missing required attributes caused the application to crash when saving the data. This has since been patched and the fix has been uploaded to the App Store, but typical review times are a week. I've requested an expedited review, but nonetheless the latest API update broke my application which was essentially using the API in the way it was intended. Please version major changes such as removal of fields (or making them optional), or at least provide some advance warning. Netshroud 07:48, 21 November 2010 (UTC)

I didn't roll the version number because I didn't consider the changes significant enough to warrant a version number change. I underestimated the impact they would have, obviously. Almost all of the API changes so far have been additional fields, so it hasn't been a problem.

What do other people using the webapi think about version number changes? Would you like to see updates roll the version number every time and leave the old version in place for a while? (At least for updates that change the web api. That's many of them, but not all of them.)

The safest way to use this (or any other web api for that matter) would be to treat most fields as optional. Particularly fields in attribute types and item types are likely to be present on only some types. That advice is a little too late to be helpful in this case though.

Joe 18:23, 23 November 2010 (UTC)

I never saw a point to versioning the requests in the first place, it just makes it to where I will have to maintain documentation for different versions when they happen. It's true that anyone using a web API should assume that all fields will be missing. I would be for getting rid of the versioning entirely. -- Lagg 02:31, 24 November 2010 (UTC)

I see no reason for having versioning, if there are major changes inbound then to my mind all that is really necessary is adequate fore-warning. It may be useful however to have additional "versions" for certain tasks; a Beta feed and a feed that just returns an example backpack with every possible significant item variation for example. It would be helpful in aiding development/testing of new content/features/changes. Ath 03:00, 24 November 2010 (UTC)

I like the idea of versioning. Nothing more annoying than finding out that your site / program has broken majorly due to a change. With versioning we can continue using the stable feed until we can make our projects compatible with the latest version. Although I'd only use it for major changes / deleting important things, not much point in increasing the version number for simple changes. I think the attribute changes would probably qualify as major. --Pc Madness 13:00, 24 November 2010 (UTC)

The three methods we provide for TF2 are simple enough that versioning hasn't really come into play. If we ever add methods that do more complex things, it is much more likely that we will need to roll the version number for changes. There is really only one argument for each method at the moment (not counting the format type.)

Next time I'm removing something I will definitely be more careful.

Joe 19:44, 24 November 2010 (UTC)

Values

float_value never matches value. In some cases (e.g. Mann Co. Supply Crates), float_value is correct and value is completely wrong. In other cases (e.g. Employee Badges), value is correct and float_value is way out. Netshroud 07:48, 21 November 2010 (UTC)

float_value is the bit pattern in value interpreted as a float. They will only match if the value is 0. When the storage for attributes changed from floating point to int not all of the attributes came along for the ride, so things are little confusing. Joe 18:26, 23 November 2010 (UTC)

GetSchema Icons

The icon for the Carouser's Capotain is still the pre-Dec 2nd version, could this be updated? Thanks. Ath 03:42, 3 December 2010 (UTC)

Fixed. I also uploaded all the icons for the TF2 public beta just in case anyone is using the webapi for that app. Joe 20:26, 3 December 2010 (UTC)

Thanks for the fix, the 128 version still seems not to be updated for me however; inquisitor_large.png is updated however. Ath 22:44, 3 December 2010 (UTC)

Looks correct to me. It may take a little time to propagate the update through the CDN, so it might be minutes to hours before you see the update. -- Joe 22:59, 3 December 2010 (UTC)

Ah, good to know. Thanks Joe. Ath 23:08, 3 December 2010 (UTC)

festive_crate.png and festive_crate_large.png are both absent, could these be added? Thanks. ~ Ath (talk) 15:56, 11 December 2010 (UTC)

The Gunboats icon[1] is bigger than the other icons, it should be cuted like the others. TheJCS 2:08, 21 December 2010 (UTC)

The Paint icons are all the same brown one. They should show the actual color of the paint, like in game. TheJCS 2:11, 21 December 2010 (UTC)

The paint icons are all generated in game, you have to create those images manually. --Pc Madness 17:42, 21 December 2010 (UTC)

Since they are different items, the API should provide different images. TheJCS 4:54, 23 December 2010 (UTC)

It's not overly difficult to create these in the same manner as the game itself does. Since the color is provided, just snag the overlay image from the gcfs (or here) and use GD or ImageMagick to color them. Here's the code I use, though there might be a cleaner way:

$i = new Imagick('path/to/paintcan.png');
$p = new Imagick('path/to/paintcan_paintcolor.png');
$m = $p->clone();
$c = new Imagick();
$c->newImage(500,500,'#'.dechex($paintcolor));
$c->compositeImage($m,imagick::COMPOSITE_COPYOPACITY,0,0);
$p->compositeImage($c,imagick::COMPOSITE_MULTIPLY,0,0);
$i->compositeImage($p,imagick::COMPOSITE_DEFAULT,0,0);

FireSlash 20:16, 24 December 2010 (UTC)

As far as I know part of the idea behind providing the icons via GetSchema was to remove the need for local storage of icons; Having to individually process or implement a system to obtain the tinted variants defeats the point somewhat. I think it's a fair request to ask for the variants to be provided. ~ Ath (talk) 22:31, 24 December 2010 (UTC)

I think it's a fair request too. It just hasn't made it to the top of my TODO list yet. -- Joe 19:46, 27 December 2010 (UTC)

Unlocking Poker Night Items

Quick question, but when you unlock a poker night item they never appear in the item feed until you start up TF2, which informs you that you have new items waiting for you. Is this a bug or are those Poker guys just doing some dodgy method of adding items to your backpack (perhaps checking if an achievement has been unlocked when TF2 is loaded?) --Pc Madness 12:12, 7 December 2010 (UTC)

I'm pretty sure when you start up TF2 is checks for a Poker Night save file to see if you have unlocked the items. If so, it grants you them. It's not a bug, it's just how it works. People have actually taken advantage of this to unlock the items without playing the game by simply logging in on a computer that has Poker Night and has unlocked the items. -- - _{(talk | contribs)} -- 12:26, 7 December 2010 (UTC)

The item backend asks Steam for this stuff when you start up TF2. You will get your items even if you're on a computer that doesn't have Poker Night installed as long as you're logged into the steam account where you earned the Poker Night achievements. How those achievements get unlocked is a question for TellTale. :) -- Joe 17:36, 21 December 2010 (UTC)

Reason for being Untradable

If it's not to much trouble, could we get the reason why an item isn't tradable? From what I've seen in game theres - Achievement Item, Purchased Item, Store Item, Promotion Item. Not really important but would be handy. :) --Pc Madness 12:12, 7 December 2010 (UTC)

At one point these flags were available (see my edit in the log that added the documentation for them), but for some reason they were removed shortly after. -- Lagg 12:58, 7 December 2010 (UTC)

Another missing image

The Prancer's Pride images are 404ing. -- Netshroud (talk | contribs) 00:48, 19 December 2010 (UTC)

The case is wrong. It works from here [2]. I'll get the GetSchema results fixed up. -- Joe 17:41, 21 December 2010 (UTC)

The icons for the three new MNC items and the Nasty weapons bundle are currently absent. ~ Ath (talk) 03:56, 25 January 2011 (UTC)

Fixed. Joe 17:04, 25 January 2011 (UTC)

Gift Content Randomly Disappearing

Joe, would it be possible to make gift content visible before the gift drops in the backpack? I've also seen some cases where the gift contents dict isn't generated at all and then will reappear again later even if it is already dropped. Naturally this has been causing problems for those that trade because optf2 falls back to the schema ID attribute if it can't find the afore mentioned dict. -- Lagg 22:27, 29 January 2011 (UTC)

I don't understand what you're asking. Is the item not dropping when unwrapping a gift, or is the item API just returning something unexpected? Joe 02:07, 1 February 2011 (UTC)

The API isn't returning what is expected. I can confirm that contained_item isn't present when the gift hasn't been dropped in the backpack proper, and it also disappears when it is in the backpack which I can't reliably reproduce, but here is a gift that at the time of this edit had no contained_item field. Other times it does. I apologize for it being hard to understand but I'm having trouble figuring it out myself. -- Lagg 04:49, 1 February 2011 (UTC)

To clarify, the contained_item is consistently never present when the gift hasn't been dropped (when it's in the "waiting to drop" queue). Once it drop in the backpack, its contained_item sometimes disappears. — Wind 04:53, 1 February 2011 (UTC)

Something strange is going on with some small percentage of gifts losing their connection to the wrapped item. (The one you linked is one of those.) It sounds like you guys have seen a contained item exist for a while and then go away, but I don't know how that would happen. Is that definitely what you're seeing? Joe 19:45, 1 February 2011 (UTC)

Yes, without a doubt. -- Lagg 19:52, 1 February 2011 (UTC)

Curiouser and curiouser. Do you know what time the contained item disappeared, approximately? Joe 20:06, 1 February 2011 (UTC)

I believe it reappeared about 2 hours after I linked to the gift but beyond that I can't say. I considered writing something to run every few minutes and check if the field is disappearing again. I can do this if it would help. -- Lagg 20:09, 1 February 2011 (UTC)

Alright the problem is showing itself again and I grabbed some API output and screenshots. The approximate time it disappeared when it was actually in my backpack was 00:31 UTC. here is the direct link and here is a dir with the screenshots and API dumps. Note how it was present in afterdrop.png but not in beforedrop.png, when I was sent the gift but did not start the game to get it, and then later when I took postdrop.png it disappeared again while the gift was in my backpack. -- Lagg 01:40, 15 February 2011 (UTC)

Also correct me if I'm wrong but it appears that the backpack specific attributes don't appear either (the medal number attribute in this case). I can provide a dump of the box's contained_item that shows this if needed. -- Lagg 21:42, 15 February 2011 (UTC)

Steam ID

Due to the way Steam IDs work, both 76561197989867376 and 76561202284834672 point to my user - the latter having m_unAccountInstance set to 2 instead of 1. ISteamUser/GetPlayerSummaries will return the correct information for both, but ITFItems_440/GetPlayerItems and ITFItems_520/GetPlayerItems return error code 8 (SteamID parameter was missing) instead of the proper (or an empty) backpack, or an appropriate error code. Probably nobody will actually run into this issue, but I figured I should mention it anyway. -- Netshroud (talk | contribs) 01:14, 1 March 2011 (UTC)