retry: Add BasicBudget

[boraarslan]

A new BasicBudget budget is added. This budget works by increasing the cost of the next retry on each successful withdraw. Cost is then decreased gradually on each successful deposit.

[seanmonstar]

Some thoughts:

• I feel that the name BasicBudget doesn't quite help convey what it does. What makes it basic? The name immediately made me think it was just a counter with no weights. This kind of looks very similar to the TpsBudget, perhaps without the window?
• What are the use cases that I would prefer using BasicBudget over TpsBudget?
• In my opinion, the TpsBudget is most likely going to be the best option, for everyone. Thus, similar to Finagle, I'd suggest we only have the one recommended budget (and possibly super simple ones like Always and Never, maybe). Including the trait allows people to use a very custom budget, but that should be for expert use. If give too many options in tower itself, we can make it hard for people who don't quite know which to use to pick the right one.
• (Not really related to this PR, sorry, but...) I think I prefer TokenBudget over the name TpsBudget.

[LucioFranco]

I feel that the name BasicBudget doesn't quite help convey what it does. What makes it basic? The name immediately made me think it was just a counter with no weights. This kind of looks very similar to the TpsBudget, perhaps without the window?

Yeah, I think the name we can play with, I think maybe something like static budget?

What are the use cases that I would prefer using BasicBudget over TpsBudget?

Ok so some context I tasked @boraarslan on this so I can try to explain. We previously had the TpsBudget which was ported from finagle iirc. While this works, the config is quite complex and not easy to set a good global default. What we want to prevent is massive amount of concurrent retries so a simple token bucket actually works quite well. Also, we have an internal implementation that is like this simpler implementation and I would like to promote this to be the default budget type and coerce users to use the default implementation. (we can talk more offline about this)

In my opinion, the TpsBudget is most likely going to be the best option, for everyone. Thus, similar to Finagle, I'd suggest we only have the one recommended budget (and possibly super simple ones like Always and Never, maybe). Including the trait allows people to use a very custom budget, but that should be for expert use. If give too many options in tower itself, we can make it hard for people who don't quite know which to use to pick the right one.

I agree, maybe we want to use an enum instead? Basically, my vision is that the default is the simpler budget but we need to keep the TpsBudget around because linkerd has it in their public config. Otherwise, I would remove the TpsBudget. I am happy to use an enum though or a trait either way.

I think I prefer TokenBudget over the name TpsBudget.

I believe both impls will have tokens so this won't work, right?

Hopefully that answers most of the questions.

[seanmonstar]

Oh, that's interesting. Would you have more of a write-up for the reasons to prefer that one? After a few glances, the configuration seems pretty equivalent in terms of complexity, and I suspect having the time window there is something we don't want to give up. But I'd be open to more data.