This article is written on 7/7/2023. It has been last updated on 19/9/2023.
Contents have been vetted to be accurate up till the date of the update as mentioned above.
This guide was written for all skill levels. For a summary, see the bolded text for each section. For feedback, feel free to post it in the comments below.
In my years of Roblox development, I have personally seen countless tutorials and user-uploaded code (or request for help with their code) regarding datastores.
While I can go down the same route as plenty of others have by giving readers a comprehensive tutorial regarding datastores, I think it would be far more interesting if I gave a slightly pessimistic cautionary tale.
Written almost entirely by (collective) experience, I hope that this guide would push people away from my footsteps and avoid making the mistakes that others and I have made.
Progress is the main factor behind good persistent engagement by your players - making tangible progress in a game is what drives your players to continue playing your game.
Conversely, no one likes losing their progress for any reason. Good datastore code is what protects the progress made by your players, and subsequently motivates players to keep playing your game.
The magnitude of this becomes drastically bigger when you mix datastores with paid products - this is when you have to be BEYOND CAREFUL with how you script datastores.
To prevent spam, all calls to datastores such as datastore:GetAsync() and datastore:SetAsync() count towards a server-wide limit, changing depending on the type of call made.
If you hit a given limit, your subsequent calls will be put into a queue. The queue will throttle the calls placed within, which may cause a delay before which they are executed.
If you hit the queue's limit however, subsequent calls will be dropped and they will throw you an ERROR instead, which may result in a failure to load or save data, or perhaps worse.
The current aforementioned limit for the queue is 30 - if you make another call if the queue is still stuck with 30 calls it hasn't executed yet, that call errors instead.
You can't get around the limits, but you can limit the data and the frequency of which you are storing and sending data. There's another technique that will help achieve this, but that will be discussed in another section.
These are the limits of datastore calls, sorted by call type:
60 + (numPlayers × 10)
5 + (numPlayers × 2)
On top of that, datastore calls are also limited by how much data you can send per minute, detailed as such:
25 MB (25 million characters) per minute
4 MB (4 million characters) per minute
All datastore entries also have a data size limit, exceeding which would cause an error. With sensible naming and updating of values however, hitting these limits would hardly be realistic.
The maximum characters you can store in each part of the datastore are:
50 characters
4,194,304 per key
Datastore, key and scope names are stored as strings, whose character lengths can be checked using with the following methods:
print(#"This is a string") -- 16
print(string.len("This is a string")) -- 16
Datastore data are also saved as strings - if they are not already a string, they will be internally converted into strings first before saving.
To check the character length of your table of data is a tad bit more complicated:
local resultString = game:GetService("HttpService"):JSONEncode({
userid = 1,
cash = 0,
pets = {},
receiptinfo = {},
}) -- returns a string.
print(#resultString) -- 48
ALL datastore calls, be it SetAsync() or ListKeysAsync(), are HTTP calls - they have a chance to be dropped for any reason, such as ping, connection failure, or a lack of response from the datastore service. Dropped datastore calls, as mentioned earlier, will result in an error.
It is wise to wrap all such calls in a pcall(), as it gives you an avenue to handle cases where data cannot be loaded or saved. More importantly, it prevents a potential error from stopping your script from running.
To use them is easy!
local success, result = pcall(function()
return DataStoreService:GetAsync(player.UserId)
)
-- success will either be true or false, depending if the function was executed properly.
-- if it ran without errors, result will be the datastore data.
-- otherwise, result will be the error message.
print(success, result)
Servers can shut down for a multitude of reasons, which will cause game:BindToClose() to fire.
Not saving data for all players in a BindToClose() means these things (all of which results in data loss):
Players.PlayerLeaving to save data when a player leaves, this event may not save data just in time before the server shuts down.Players.PlayerAdded will not save data just in time too.The best practice in this case, is to save data in a BindToClose() event as well, since this almost guarantees that a player's data will be saved even in those edge cases above.
game:BindToClose(function()
for _, player in Players:GetPlayers() do
local _, _ = pcall(function()
datastore:SetAsync(player.UserId, data[player.UserId])
end)
end
end)
Caching data in this case means to "save" whatever data belonging to a player into a container like a ModuleScript, assuming it is put into ServerStorage for security reasons.
Some benefits to this include:
Caching data, and subsequently working with data in that cache means you will not have to set data into datastores as frequently, mitigating the chances you will hit a datastore limit.
By extension, it also means there is almost no chance for error when updating data due to bad HTTP requests.
It is also generally faster than HTTP calls, even if that cache has to be modified by Remotes.
Datastore data only allows primitive datatypes, such as strings, booleans and numbers.
To save a datatype such as Vector2, Color3, and others, serialize them in a format that the datastore would understand.
The simplest way to do this is to convert them into arrays to be saved - when they are loaded, simply get those data and create the needed value.
-- for brevity, pcalls() will not be used here.
-- all datastore calls are also assumed to run without error.
-- to "save" a color:
local ineligible = Color3.new(1, 1, 1) -- this cannot be saved into datastores...
local eligible = { ineligible.R, ineligible.G, ineligible.B } -- but this can!
datastore:SetAsync("color", eligible)
-- to create a color with the saved values:
local result = datastore:GetAsync("color")
local color = Color3.new(result[1], result[2], result[3])
-- OR
local color = Color3.new(table.unpack(result))
This extends to the data itself; to conserve key space, you may wish to discard dictionary keys, so that the pending data would become an array.
This has to be done within reason however, as the resulting table might be unreadable and too difficult for you to deserialize when you get the data back.
First-time users will always start with no data (nil) in datastores.
If you try to get data from a first-time user the normal way, you'll end up getting nil instead.
To solve this, have a set of default data values that you can hand out to players; or statements are your best friends in this case.
-- for brevity, pcalls() will not be used here.
-- all datastore calls are also assumed to run without error.
local data = datastore:GetAsync(userid)
local CashValue = data.cash or 0 -- if data.cash is nil, set it to 0 instead.
local ExpValue = data.xp or 50 -- if data.exp is nil, set it to 50 instead.
This may also help in cases where the player's data does exist in datastores, but cannot be loaded for any reason.
Datastore calls may introduce a (not so) subtle bug where a player's data is loaded first, even if the game is still busy saving the data for that player's previous gaming session.
This is primarily the reason why duplication bugs exist in Roblox games, and is formally known as a race condition, an inherent problem with all HTTP calls.
While this may seem unrealistic on the surface, the problem becomes obvious in cases like:
A way to solve this issue is to bind the server's ID to the player's data whenever they load into the server, using game.JobId, and to set it to 0 if the user's data is done saving.
local dataset = {
serverid = game.JobId
cash = 0,
xp = 50,
}
If the player loads in while serverid is not 0, you'll know that the player's data has not been saved yet, and you can block the load call and handle the player's game session accordingly.
If you've actually read up to this point (and without doing a TL;DR):
If you have actually got to this point without trouble, I genuinely believe you have the technical skills to understand what's next.
While GetAsync() and SetAsync() are sufficient for most use cases, they have their own issues that may manifest themselves as problems that I have talked about in the above section.
GetAsync() has a 4-seconds local cache, where any subsequent GetAsync() calls made within that 4 seconds period will refer to that cache instead.
When a data value gets updated/overwritten to the datastore, it will still take a moment for the relevant caches to reflect the new value. This will also introduce race conditions.
Internally, GetAsync() calls are most of the time faster than SetAsync(). This is also another source of race conditions, and is actually the main cause of dupe bugs.
Cross-server games get hit by this the most.
SetAsync() will NOT return you the updated value after you call it; to do that, you'll have to call GetAsync().
That's 2 separate HTTP calls, which means twice the chance of an error which may prevent this from being done properly.
To solve these headaches, I strongly recommend using UpdateAsync() in substitute of all GetAsync() and SetAsync() calls, for the following benefits:
UpdateAsync() calls are "queued" - subsequent calls will not execute until the previous one has actually been completed, EVEN ACROSS SERVERS. This removes the possibility of a race condition.UpdateAsync() also returns you the updated value, with no caching to speak of. This means that setting data using UpdateAsync() will also get you the resulting data, every time.UpdateAsync() calls can be checked and subsequently canceled by returning nil.When used in conjunction with session locking, dupe bugs become next to impossible to achieve. I promise.
To get and set data using UpdateAsync() is actually pretty easy.
If it helps you out, here's some sample code:
-- for brevity, pcalls() will not be used here.
-- all datastore calls are also assumed to run without error.
-- to get data:
local result = nil
datastore:UpdateAsync("key", function(old)
result = old
return nil
end)
-- to set data:
local new = {}
local result = datastore:UpdateAsync("key", function(old)
return new
end)
-- to cancel the write operation:
local result = datastore:UpdateAsync("key", function(old)
return nil
end)
If you only intend to get the latest data, be sure to cancel the write operation, to save on bandwidth and time.
UpdateAsync()does not work for OrderedDataStores.
UpdateAsync()is also most effective if you're saving data in tables.
Note: This section is not sponsored, although I will be glad if it is.
If all these sound too scary for you to implement alone, ProfileService helps by handling all that for you, and has already seen use in many games, big or small.
This is basically the only library I recommend these days, and for good reason.
If anything, many pointers that I have discussed here are things that ProfileService has already done for you, so that you won't have to.
Give it a shot sometime.
Credits go to
lolerisfor the making ofProfileService.