Reduce Default Sampling Nb. Records from 10000
Hi,
I am just wondering on a user-level am I able to reduce the default number of records queried for when sampling a dataset from the 10k default to a custom amount? Often I want to adjust the sampling parameters or filters and waiting for 10k records to load before doing that is time wasted.
Thanks!
Best Answer
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
Hi @joe88
Thanks for this clarification. Currently, there is no setting to control this globally. Instance-wide you can only control memory usage allowed for preview under Administration->Settings->Resource Control.
Answers
-
Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
Hi @joe88
You will be able to retrieve as many rows as you set on the sampling panel. DSS will add a limit statement to a query to get the data.
Reducing shouldn't be a problem while increasing may hit max memory usage reserved for the sampling.
Are you saying that you see that DSS still queries 10000 even when you change the sampling limit?
-
I was wondering if there was an application-wide setting that I could apply to adjust it from 10k default to something like 5k or 2k records sampled initially. Something that would apply for my user account across all datasets that I open.