CIP Proposal 18 - Configuration flag to remove API query limit defaults


#1

This is the discussion thread for the proposed CIP 18 to remove API query limits by configuration flags.

Link to the CIP: https://github.com/CounterpartyXCP/cips/blob/cip18/cip-0018.md

Please, discuss.


#2

Overall I think making API limit a configurable option is a good idea.

Adding an API_LIMIT_1000 flag in config.py, making it configurable from the counterparty-cli code, and acting on its setting in api.py.

Rather than a boolean flag to indicate a limit of 1000 or no limit, I would prefer a API_LIMIT flag which can be configured to any integer, with 0 indicating no limit .


#3

This could make sense, would there be any issue limiting queries below 1000? wouldn’t that be backwards incompatible? (software should be aware then that there’s a limit different than expected, maybe get that info on get_running_info call?)


#4

Changed the CIP to account for @jdogresorg suggestion, which seems reasonable.


#5

@chiguireitor ch What would the default API_LIMIT_ROWS be in the config file?
You mentioned in the CIP that care needs to be take so that no default configuration makes the node susceptible to DDOS attacks


#6

IMO the default should be 1000 (same as it is now). The new feature is the ability to customize that limit or disable it entirely (set value to 0).


#7

Indeed, default should be 1000 to allow non-configured-for-this-feature instances to continue working the same


#8

A CIP bounty address has been requested for CIP-18.

Address 13CoTjbVaYzg9kQ2arjKcCmzT5WZmRfgdq has been provisioned to hold donated funds for CIP-18


#9

Initial commits for CIP18 support

Testing underway


#10

What should be the expected behaviour for a server with a limit lower than 1000 with a non-limited call? silently send the limited amount and pray for the best or fail fast so developers get informed about the non-standard limits (current coded behaviour)?


#11

PRs merged to master:

Milestone #1 achieved (although merged to master, won’t ask milestone #3 after testing #2) https://github.com/CounterpartyXCP/cips/blob/master/cip-0018.md


#12

Installing mainnet server for the test atm. Sync up to block ~423k right now


#13

Test server with on mainnet has been deployed and synced up (finally)

http://mg.contimita.com:4000

Username/Password: rpc/1234

Current api limit rows: 0

Limit should be specified explicitly in calls to prevent standard default behaviour (if not specified in the call, a limit of 1000 is applied).

Because of the size of some calls, unbounded queries over a non-local uplink will take a good time to complete, be sure about what you want to query before putting “limit”: 0

Also, just to note… some calls like “get_issuances” with limit 0 are gona DoS the server for a very long time while replying stuff… if it happens hit me up here or in the telegram channel (same nick as here)


#14

I did a few simple tests using get_issuances and get_balances and passing 0 or a limit over 1000 returns more than 1000 results as expected.

get_issuances API call with 0 limit returns over 1000 results

get_balances API call with 0 limit returns over 1000 results


#15

Ok, restarting the xcp server with limit set to 100


#16

Default behavior is to error out because of the lower limit.


#17

Found a bug: Setting limit to 0 on a limited or default server returns the whole set.


#18

Bugfix pushed to master


#19

Seems like the feature is done, slated for release on the next milestone (you can also pull from master to get the latest).