Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
How can I change the default location where a .conf file is created to any custom location?
Added on January 15, 2025 8:57AM
Likes: 0
Replies: 2
Efthymios
Registered Posts: 1 ✭
Hello Community,
I am using the great expectations package in a dss project for my Data Quality checks.
I have already installed it in my code env and using it in a python script for the time being.
Even though, the package properly runs in a python notebook when I save it back to recipe I get the following error:
"
Job failed: Error in Python process: At line 15: <class 'PermissionError'>: [Errno 13] Permission denied: '/home/dataiku/.great_expectations/great_expectations.conf'
"
I think the error occurs because the package is creating .conf file in my dss user home , I need to change the default location to any custom location. Can you please help me on how to override the default location of the 'great_expectations.conf' file to another directory outside of the dss user's HOMEDIR ?
Thanks in advance,
Efthymios D.
Operating system used: Windows 10 Enterprise
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,363 NeuronThis is not a Dataiku question but a great expectations package question so probably best to raise as a discussion in the great_expectations/discussions Github repo. It does look like you can pass the configuration values as environment variables or as parameters to get_context():
-
After trying everything, none of the parameters really work. The module still checks inside /home/dataiku/ directory and if you don't have access, you will still get an error.
This is a quick not-so-clean fix I made to get things done. You can put it in a function inside libraries to declutter your code.
I understand this is out of the scope of this community, I just hope this helps somebody.
#great-expectations==1.3.5 import pathlib import great_expectations as gx from great_expectations.data_context import AbstractDataContext YOUR_CONF_PATH = pathlib.Path("SET/YOUR/PATH/HERE…") # use a managed folder for example AbstractDataContext._ROOT_CONF_DIR = YOUR_CONF_PATH / ".great_expectations"
AbstractDataContext._ROOT_CONF_FILE = AbstractDataContext._ROOT_CONF_DIR / "great_expectations.conf" context = gx.get_context(mode="file")