1158

What is an idempotent operation?

1

18 Answers 18

1284

In computing, an idempotent operation is one that has no additional effect if it is called more than once with the same input parameters. For example, removing an item from a set can be considered an idempotent operation on the set.

In mathematics, an idempotent operation is one where f(f(x)) = f(x). For example, the abs() function is idempotent because abs(abs(x)) = abs(x) for all x.

These slightly different definitions can be reconciled by considering that x in the mathematical definition represents the state of an object, and f is an operation that may mutate that object. For example, consider the Python set and its discard method. The discard method removes an element from a set, and does nothing if the element does not exist. So:

my_set.discard(x)

has exactly the same effect as doing the same operation twice:

my_set.discard(x)
my_set.discard(x)

Idempotent operations are often used in the design of network protocols, where a request to perform an operation is guaranteed to happen at least once, but might also happen more than once. If the operation is idempotent, then there is no harm in performing the operation two or more times.

See the Wikipedia article on idempotence for more information.


The above answer previously had some incorrect and misleading examples. Comments below written before April 2014 refer to an older revision.

15
  • 25
    Example :since answer above states that Idempotent operations are often used in the design of network protocols here's a related example **GET is not suppose to change anything on the server, so GET is, idempotent. In HTTP/servlet context, it means the same request can be made twice with no negative consequences. **POST is NOT idempotent.
    – KNU
    Apr 1, 2014 at 10:37
  • 4
    Is "stateless" synonymous with "idempotent"? Dec 5, 2014 at 17:09
  • 5
    @MichaelOsofsky: No, in the Python set example in the answer, the set object clearly has state and also offers some idempotent operations such as discard. Dec 7, 2014 at 19:44
  • 2
    @MichaelOsofsky, discard can also be implemented in a stateless way by encompassing the state in the return value: discard([my_set, x]) = [my_new_set, x]. So you can do discard(discard([my_set, x])). Note that [my_new_set, x] is just one argument and its type is 2-tuple.
    – Pacerier
    Mar 10, 2015 at 22:59
  • 9
    @Green When using the term same effect in the context of impotency, it means that the result is the same, not the action. Calling discard(x) a second time will have the same effect as calling it the first time: The set will no longer contain x. Computing idempotence is about the robustness of a system. Since things can fail (e.g. network outage), when a failure is detected, how do you recover? The easiest recovery is to just do it again, but that only works if doing it again is idempotent. E.g. discard(x) is idempotent, but pop() is not. It's all about error recovery.
    – Andreas
    Apr 27, 2016 at 15:27
178

An idempotent operation can be repeated an arbitrary number of times and the result will be the same as if it had been done only once. In arithmetic, adding zero to a number is idempotent.

Idempotence is talked about a lot in the context of "RESTful" web services. REST seeks to maximally leverage HTTP to give programs access to web content, and is usually set in contrast to SOAP-based web services, which just tunnel remote procedure call style services inside HTTP requests and responses.

REST organizes a web application into "resources" (like a Twitter user, or a Flickr image) and then uses the HTTP verbs of POST, PUT, GET, and DELETE to create, update, read, and delete those resources.

Idempotence plays an important role in REST. If you GET a representation of a REST resource (eg, GET a jpeg image from Flickr), and the operation fails, you can just repeat the GET again and again until the operation succeeds. To the web service, it doesn't matter how many times the image is gotten. Likewise, if you use a RESTful web service to update your Twitter account information, you can PUT the new information as many times as it takes in order to get confirmation from the web service. PUT-ing it a thousand times is the same as PUT-ing it once. Similarly DELETE-ing a REST resource a thousand times is the same as deleting it once. Idempotence thus makes it a lot easier to construct a web service that's resilient to communication errors.

Further reading: RESTful Web Services, by Richardson and Ruby (idempotence is discussed on page 103-104), and Roy Fielding's PhD dissertation on REST. Fielding was one of the authors of HTTP 1.1, RFC-2616, which talks about idempotence in section 9.1.2.

9
  • 22
    "idempotence" is a heavily overloaded word because it sounds grandiloquent and has enough characters to pass the sesquipedalian check. If Benjamin Peirce had chosen a simpler sounding word, we wouldn't even have this question today.
    – Pacerier
    Mar 11, 2015 at 11:15
  • 3
    How to understand it: Similarly DELETE-ing a REST resource a thousand times is the same as deleting it once? You cannot delete the resource again if it is already deleted.
    – Green
    Oct 10, 2015 at 8:14
  • 3
    @Green but you don't delete it the first time. You send a delete request. The important point is that you can send as many requests as you like.
    – Caleth
    Apr 6, 2017 at 12:54
  • 2
    @JimFerrans I see. I thought there might be some functionality-related reason (built into the HTTP itself) of why PUT can be resent without worries while POST cannot. Now it appears that we are simply required to conform to the HTTP standards and the behaviour is totally based on how the server is implemented
    – mangusta
    Feb 22, 2020 at 6:11
  • 1
    "In arithmetic, adding zero to a number is idempotent." This is a bad example because adding zero doesn't change anything. Being idempotent means that it (probably) changes things, but in the same way whether it is applied one or multiple times. The abs() function given further above is a better example.
    – dret
    Nov 19, 2021 at 11:07
151

No matter how many times you call the operation, the result will be the same.

8
  • 9
    I've heard idempotent defined as either or both of the below: 1) For a given set of inputs it will always return the same output. 2) Does not produce any side effects. My question is, if a function conforms to #1, but not #2, because it results in a side effect unrelated to the computation (logs the request to a data store, for example), is it still considered idempotent? Jun 28, 2012 at 22:32
  • 13
    The result of calling an operation must include the state of the system, so if the operation has some cumulative side effect it is not idempotent; however, if the side effect leaves the system in the same state no matter how many times the operation is called, then it may be idempotent.
    – Robert
    Jul 17, 2012 at 21:45
  • 9
    Short and sweet, I love that kind of answer. Not sure why I have to look this term up constantly, it's one that just doesn't stay with me.
    – Prancer
    Feb 2, 2015 at 12:59
  • 1
    @KeithBennett, The second definition is wrong. "No side effect" does not mean idempotent. Idempotent functions can have side effects. E.g. MySQL's truncate and delete.
    – Pacerier
    Mar 10, 2015 at 23:11
  • 1
    so if there is a row counter in database, which is used to check data integrity of the request input vs. database state, does that row counter need to be considered in the idempotent definition? that row counter will increase for every request but not being returned as part of result.
    – swcraft
    Apr 27, 2020 at 15:12
64

Idempotence means that applying an operation once or applying it multiple times has the same effect.

Examples:

  • Multiplication by zero. No matter how many times you do it, the result is still zero.
  • Setting a boolean flag. No matter how many times you do it, the flag stays set.
  • Deleting a row from a database with a given ID. If you try it again, the row is still gone.

For pure functions (functions with no side effects) then idempotency implies that f(x) = f(f(x)) = f(f(f(x))) = f(f(f(f(x)))) = ...... for all values of x

For functions with side effects, idempotency furthermore implies that no additional side effects will be caused after the first application. You can consider the state of the world to be an additional "hidden" parameter to the function if you like.

Note that in a world where you have concurrent actions going on, you may find that operations you thought were idempotent cease to be so (for example, another thread could unset the value of the boolean flag in the example above). Basically whenever you have concurrency and mutable state, you need to think much more carefully about idempotency.

Idempotency is often a useful property in building robust systems. For example, if there is a risk that you may receive a duplicate message from a third party, it is helpful to have the message handler act as an idempotent operation so that the message effect only happens once.

2
  • 1
    If for pure functions f(x) = f(f(x)), Do you mean that f(x){return x+1;} is not a pure function? because f(x) != f(f(x)): f(1) gives 2 while f(2) gives 3.
    – Pacerier
    Mar 10, 2015 at 22:08
  • 4
    @Pacerier No, @mikera is saying pure and idempotent implies f(x) = f(f(x)). But as @GregHewgill mentioned, in order for this definition to make sense, you have to consider x as an object and f as an operation that mutates the state of the object (ie: the output of f is a mutated x). Oct 17, 2016 at 16:14
39

A good example of understanding an idempotent operation might be locking a car with remote key.

log(Car.state) // unlocked

Remote.lock();
log(Car.state) // locked

Remote.lock();
Remote.lock();
Remote.lock();
log(Car.state) // locked

lock is an idempotent operation. Even if there are some side effect each time you run lock, like blinking, the car is still in the same locked state, no matter how many times you run lock operation.

1
  • 2
    A good idea would be to contrast this with some remotes, which instead of two buttons lock() and unlock(), have one button toggleLock(). In that case, clicking the button is not idempotent - every click ends up changing the state, alternating between unlocked and locked.
    – Stobor
    May 13, 2022 at 8:06
30

An idempotent operation produces the result in the same state even if you call it more than once, provided you pass in the same parameters.

2
21

An idempotent operation is an operation, action, or request that can be applied multiple times without changing the result, i.e. the state of the system, beyond the initial application.

EXAMPLES (WEB APP CONTEXT):

IDEMPOTENT: Making multiple identical requests has the same effect as making a single request. A message in an email messaging system is opened and marked as "opened" in the database. One can open the message many times but this repeated action will only ever result in that message being in the "opened" state. This is an idempotent operation. The first time one PUTs an update to a resource using information that does not match the resource (the state of the system), the state of the system will change as the resource is updated. If one PUTs the same update to a resource repeatedly then the information in the update will match the information already in the system upon every PUT, and no change to the state of the system will occur. Repeated PUTs with the same information are idempotent: the first PUT may change the state of the system, subsequent PUTs should not.

NON-IDEMPOTENT: If an operation always causes a change in state, like POSTing the same message to a user over and over, resulting in a new message sent and stored in the database every time, we say that the operation is NON-IDEMPOTENT.

NULLIPOTENT: If an operation has no side effects, like purely displaying information on a web page without any change in a database (in other words you are only reading the database), we say the operation is NULLIPOTENT. All GETs should be nullipotent.

When talking about the state of the system we are obviously ignoring hopefully harmless and inevitable effects like logging and diagnostics.

18

Just wanted to throw out a real use case that demonstrates idempotence. In JavaScript, say you are defining a bunch of model classes (as in MVC model). The way this is often implemented is functionally equivalent to something like this (basic example):

function model(name) {
  function Model() {
    this.name = name;
  }

  return Model;
}

You could then define new classes like this:

var User = model('user');
var Article = model('article');

But if you were to try to get the User class via model('user'), from somewhere else in the code, it would fail:

var User = model('user');
// ... then somewhere else in the code (in a different scope)
var User = model('user');

Those two User constructors would be different. That is,

model('user') !== model('user');

To make it idempotent, you would just add some sort of caching mechanism, like this:

var collection = {};

function model(name) {
  if (collection[name])
    return collection[name];

  function Model() {
    this.name = name;
  }

  collection[name] = Model;
  return Model;
}

By adding caching, every time you did model('user') it will be the same object, and so it's idempotent. So:

model('user') === model('user');
1
  • 1
    This answer doesn't seem right. Idempotence is about calling an operation that you expect to change some state, given the input parameters, which if you then call again with the same parameters has no further effect on state. But in your example, before even using caching, if we call model('user') twice, this is already idempotent, there is no change of state, just creation and return of a new distinct object to the caller (not stored). You are describing a object identity factory pattern that ensures 'the same' object is returned across calls, useful but not explaining idempotence.
    – Ash
    Jul 2, 2020 at 4:12
11

Quite a detailed and technical answers. Just adding a simple definition.

Idempotent = Re-runnable

For example, Create operation in itself is not guaranteed to run without error if executed more than once. But if there is an operation CreateOrUpdate then it states re-runnability (Idempotency).

1
  • 5
    This is a deceptive definition. re-runnability does not guarantee to be idempotent. An operation can be re-runnable and in each run it can add additional effects to the result so it would not be idempotent. Apr 16, 2018 at 14:38
10

It is any operation that every nth result will result in an output matching the value of the 1st result. For instance the absolute value of -1 is 1. The absolute value of the absolute value of -1 is 1. The absolute value of the absolute value of absolute value of -1 is 1. And so on.

See also: When would be a really silly time to use recursion?

0
10

Idempotent Operations: Operations that have no side-effects if executed multiple times.
Example: An operation that retrieves values from a data resource and say, prints it

Non-Idempotent Operations: Operations that would cause some harm if executed multiple times. (As they change some values or states)
Example: An operation that withdraws from a bank account

1
  • 6
    Actually a wrong answer! for the Idempotent operation saying "have no side-effects" is not right. for the non-idempotent operations saying " cause some harm" is a confusing answer. Apr 16, 2018 at 15:37
8

An idempotent operation over a set leaves its members unchanged when applied one or more times.

It can be a unary operation like absolute(x) where x belongs to a set of positive integers. Here absolute(absolute(x)) = x.

It can be a binary operation like union of a set with itself would always return the same set.

cheers

1
  • 2
    An idempotent operation is one where f(f(x)) = f(x). "leaves its members unchanged" is not a right answer. Apr 16, 2018 at 15:44
5

In short, Idempotent operations means that the operation will not result in different results no matter how many times you operate the idempotent operations.

For example, according to the definition of the spec of HTTP, GET, HEAD, PUT, and DELETE are idempotent operations; however POST and PATCH are not. That's why sometimes POST is replaced by PUT.

0
4

An operation is said to be idempotent if executing it multiple times is equivalent to executing it once.

For eg: setting volume to 20. No matter how many times the volume of TV is set to 20, end result will be that volume is 20. Even if a process executes the operation 50/100 times or more, at the end of the process the volume will be 20.

Counter example: increasing the volume by 1. If a process executes this operation 50 times, at the end volume will be initial Volume + 50 and if a process executes the operation 100 times, at the end volume will be initial Volume + 100. As you can clearly see that the end result varies based upon how many times the operation was executed. Hence, we can conclude that this operation is NOT idempotent.

I have highlighted the end result in bold.


If you think in terms of programming, let's say that I have an operation in which a function f takes foo as the input and the output of f is set to foo back. If at the end of the process (that executes this operation 50/100 times or more), my foo variable holds the value that it did when the operation was executed only ONCE, then the operation is idempotent, otherwise NOT.

foo = <some random value here, let's say -2>

{ foo = f( foo ) }   curly brackets outline the operation

if f returns the square of the input then the operation is NOT idempotent. Because foo at the end will be (-2) raised to the power (number of times operation is executed)

if f returns the absolute of the input then the operation is idempotent because no matter how many multiple times the operation is executed foo will be abs(-2).
Here, end result is defined as the final value of variable foo.


In mathematical sense, idempotence has a slightly different meaning of:
f(f(....f(x))) = f(x)
here output of f(x) is passed as input to f again which doesn't need to be the case always with programming.

1
2

1. Basics Of Idempotence

Mathematically, a function is said to be idempotent if f(f(x)) = f(x). So, if we have a function absolute(x), which returns the positive counterpart of an integer x; then the following MUST be true:

absolute(absolute(x)) = absolute(x)

If we take x as -100, nomatter how many times we repeat this operation, the result after the first execution and the result after the first execution would exactly be the same.

So,

absolute(-100) = 100
AND
absolute(absolute(-100)) = absolute(100) = 100
=> absolute(absolute(x)) = absolute(x)

❗ An important point to note here is that the "result" doesn't necessarily mean the data returned from the function. It refers to the data on which the operation is being performed.

Therefore, If you take a function in object oriented programming, for example, a Set with values {1,2,4} and you try to do add(2); the RESULT i.e. the contents of the set will NOT change. No matter how many times you repeat this operation the result will always be the same i.e. {1,2,4}. However, when you add an element which doesn't exist in the set i.e. you do a add(3) the set becomes (not taking order into account) {1,2,4,3}. But when you repeat the operation again (even 100 times), the result (i.e. the set) will continue to contain {1,2,4,3}. This is not true for a List, because it can contain duplicates. And each time you add something to a List, it will always be added.

To simplify,

An operation is said to be Idempotent if it can be repeated multiple times in such a way that the result after the first execution and result after n executions is exactly the same.

Adding on to this, any operation which doesn't change the data, is Idempotent because it doesn't have any impact on the state of the data.

2. Idempotence in REST So, request methods like GET, HEAD, OPTIONS, TRACE.. (which don't change the data) are Idempotent (provided they are implemented correctly). For the methods which intend to change the data, their Idempotency can vary. For example:

  • PUT & DELETE: Should be implemented in such a way that they are Idempotent.
  • POST and PATCH: Not guaranteed to be Idempotent. They might be or they might not be.

3. Why PUT MUST be Idempotent?

According to RFC-7231 by IETF, PUT is intended to replace the state of the server. So, if you have an address stored on a website, the PUT request should have the whole address in its request. Furthermore, it should replace the data on the server (or create if it doesn't already exist). So, the RESULT after first execution of the request and result after multiple executions of the same request is exactly the same.

4. Why DELETE MUST be Idempotent?

Delete is meant to delete a resource on the server. So, if you're deleting your address, the same address cannot be deleted again (it doesn't exist and nothing further gets deleted). So again, the RESULT after the first execution and RESULT after multiple executions of the same request is exactly the same.

❗ A common misconception in REST is that if a request receives the same response every time it is executed, it is Idempotent. This is incorrect. The result always relates to the data on the server (i.e. the data on which the operation is being performed on). When thinking about Idempotency in REST, think about the state of the server.

This video explains these concepts quite in detail: https://www.youtube.com/watch?v=R7s2FVN4c9Q

1

my 5c: In integration and networking the idempotency is very important. Several examples from real-life: Imagine, we deliver data to the target system. Data delivered by a sequence of messages. 1. What would happen if the sequence is mixed in channel? (As network packages always do :) ). If the target system is idempotent, the result will not be different. If the target system depends of the right order in the sequence, we have to implement resequencer on the target site, which would restore the right order. 2. What would happen if there are the message duplicates? If the channel of target system does not acknowledge timely, the source system (or channel itself) usually sends another copy of the message. As a result we can have duplicate message on the target system side. If the target system is idempotent, it takes care of it and result will not be different. If the target system is not idempotent, we have to implement deduplicator on the target system side of the channel.

1
  • Idempotency of single requests sent in isolation from any other requests (or anything else happening that changes the state of the system), is not the same as reordering requests. A HTTP PUT request and a HTTP DELETE request should both be individually idempotent - but that does not mean that the order of calling PUT and DELETE on the same URL does not matter, because the PUT request might have side effects! Mar 8, 2016 at 9:25
0

For a workflow manager (as Apache Airflow) if an idempotency operation fails in your pipeline the system can retry the task automatically without affecting the system. Even if the logs change, that is good because you can see the incident.

The most important in this case is that your system can retry the task that failed and doesn't mess up the pipeline (e.g. appending the same data in a table each retry)

1
  • but simply explain "idempotency operation" first before the above. Jul 21, 2022 at 10:08
0

Let's say the client makes a request to "IstanceA" service which process the request, passes it to DB, and shuts down before sending the response. since the client does not see that it was processed and it will retry the same request. Load balancer will forward the request to another service instance, "InstanceB", which will make the same change on the same DB item.

We should use idempotent tokens. When a client sends a request to a service, it should have some kind of request-id that can be saved in DB to show that we have already executed the request. if the client retries the request, "InstanceB" will check the requestId. Since that particular request already has been executed, it will not make any change to the DB item. Those kinds of requests are called idempotent requests. So we send the same request multiple times, but we won't make any change

Not the answer you're looking for? Browse other questions tagged or ask your own question.