Where to find Weed, coke, shards, MD caps, Xtc, Keta, mushies, Oxys plug in Calgary Canada

Where to find Weed, coke, shards, dexies, vyvanse, MD caps, Xtc, Vallies, Keta, mushies, Oxys plug in Newcastle Australia

Python based templating engines like jinja and mako allow users to 'print' text in various formats, using conditional logic statements like if-else and for loops. I think dataiku should offer an off the shelf Python based templating engine that would allow users to upload their template(s) and pass a `context dict` to the templating engine to apply. The templating engine would be a recipe in and of itself where users could define a context dictionary in JSON like variables. Or they could add dataset inputs that they could preprocess with Python to generate their context dict. The output would the printed files, either as standalone files or as a dictionary of strings with the key being the name of the printed file and the value being the printed text.

thx

It would be great if the connections were sorted as they are in most other areas of DSS. For example, it would greatly improve the user experience if the connections were sorted by Type, Credential entered, and Connection.

The column headers appear to have a sort option but it appears to be inactive. Enabling user selected sorting would be another means of improving the user experience although just sorting the list as described above would be perfectly sufficient.

I have an issue while constructing excel with python recipe.
I need to use precise packages (xlsxwriter, xlwings, openpyxl and so on) that work very well in Anaconda.But in Dataiku python recipe it's not the case.
In my mind with python recipe I could do whatever python usually does.

Ex below of a line that does not work:

XlwingsError: Your platform only supports the instantiation via xw.Book(json=...)

Does anyone have a solution please ?

Thank you!

Best regards

I recently worked on a Git integration in our Dataiku Designer instance and would like to share a few trivial insights here post the resolution with a Dataiku support team member.

Shared two helpful references while working on the integration:

• Cloning a Library from a Remote Git Repository
• Working with Git

To summarize the main steps involved:

1. Switched to the user role (e.g. dssuser) when it was used to install the DSS instance (this step could avoid lots of potential issues in the following steps)
2. ssh-keygen to generate a ssh key and it should save the outcome under the path /home/dssuser/.ssh/
3. Copied and pasted the key value from a public key provided from step 2 into a desired Git repo.
4. Within Admin UI (Administration/Settings/Git), added a corresponding Git repo URL within the whitelist param.
   • Additionally, one could further modify the "Configuration options" param, depending on the use cases, such as the above documentation pointed out to point with a different key path and have a finer security control by user groups.
   • Make sure it's ticked within the box of "Let DSS control SSH command"; otherwise, it won't ask your dssuser role to execute SSH commands.

     ---

     Operating system used: Linux (ubuntu 20.04)

Dataiku prompts this error

ValueError: I/O operation on closed file.

when I tried to execute the script below:

with folder.get_download_stream(path_csv) as f:
data = pd.read_csv(f, encoding='latin-1')

The reason I put that encoding is due to the utf-8 error if encoding is not specified (

'utf-8' codec can't decode byte 0xe1 in position 16: invalid continuation byte

).

Any idea how I can solve this?

Operating system used: Windows 10

As shown in the image below, I have several Excel files stored in the Sharepoint site, my goal is to import all the files and later stack them into one dataset, is this possible? How, please.
There are too many files that I can't import individually, and every week two files are added, so I want to automate this.

Thanks for any help.

Operating system used: Windows

1. Summary of prediction (# datapoints, MAPE, MAD, under and over predictions)
2. Future dataset with prediction

Has anyone attempted to create similar dashboards in the past? Are there any resources available for me to refer to? What is the effort required to set up this plugin?
Ultimately, I want to download these HTML sheets and share them with the stakeholders.

Operating system used: Windows

Operating system used: Windows

But now, with a Python script i want to parse this sharepoint, in order to list the available files, and apply another python script to one of them.

When i create a Python Receipe , it creates a header "Dataset' like:

InputDLISSharedpoint = dataiku.Dataset("InputDLISSharedpoint")
InputDLISSharedpoint_df = InputDLISSharedpoint.get_dataframe()

How can i parse this sharepoint to list all the files?

Thank you

We actively leverage cluster models with partitions for various use cases: from segmenting field representatives for numerous regions/countries, to identifying patient sub-groups for different populations in a therapy area, and so on.

By supporting partitioning for visual ml clustering models this will help expedite scaling model creation across the enterprise and make available the solution to additional users who aren't as familiar with python code.

I have about 1500 people to schedule into 8-10 events (shifts), with participant availability constraints, event capacity constraints, and possibly some other constraints I’m not yet aware of.

I’m looking for any help in kick starting this project.

Operating system used: Linux

1st Snapshot: Flow maintained in Parent project

2nd Snapshot: Usage of plugin in Usage project.

3rd Snapshot: Flow executed in Usage project (enabled Debug application in Plugin)

Error that we are getting:

Error in Python process: At line 11: <class 'Exception'>: Managed folder D1****H* cannot be used : declare it as input or output of your recipe

Explanation:
Here, D1****H* is the original folder of the project which got converted to Plugin. Ideally, this should have been the reference to manage folder of Usage project.

Plugin Configuration:

{
   "projectExportManifest": {
      "exportUploads": false,
      "exportAllInputDatasets": false,
      "exportAllDatasets": false,
      "exportManagedFS": false,
      "exportAnalysisModels": false,
      "exportSavedModels": false,
      "exportModelEvaluationStores": false,
      "exportProjectResources": false,
      "exportLabelingTasks": false,
      "exportManagedFolders": false,
      "exportAllInputManagedFolders": false,
      "exportInsightsData": false,
      "includedDatasetsData": [],
      "includedSavedModels": [],
      "includedManagedFolders": [],
      "includedModelEvaluationStores": [],
      "includedCodeStudios": [],
      "includedLabelingTasks": [],
      "exportGitRepository": false
   },
   "instanceFeatures": {
      "showFlowNavLink": false,
      "showLabNavLink": false,
      "showCodeNavLink": false,
      "showSwitchToProjectViewButton": true,
      "showVersionControlFeatures": false
   },
   "useAppHomepage": true,
   "homepageSections": [],
   "id": "PROJECT_POC*********",
   "label": "POC - *******",
   "shortDesc": "",
   "tags": [],
   "useAsRecipeSettings": {
      "icon": "icon-puzzle-piece",
      "category": "Custom",
      "inputRoles": [
         {
            "objectId": "D1****H*",
            "roleLabel": "Input",
            "type": "MANAGED_FOLDER",
            "$touched.it.roleLabel": true,
            "$invalid": false
         }
      ],
      "outputRoles": [
         {
            "objectId": "X****J*t",
            "roleLabel": "Output",
            "type": "MANAGED_FOLDER",
            "$touched.it.roleLabel": true,
            "$invalid": false
         }
      ],
      "variablesEditionTile": {
         "behavior": "MODAL",
         "params": []
      },
      "runScenarioTile": {
         "scenarioId": "RUNE2E",
         "buttonText": "Run E2E"
      }
   },
   "instantiationPermission": "USE_APP_MASTER_PERMISSIONS",
   "limitedVisibilityEnabled": "ENABLED",
   "accessRequestsEnabled": "ENABLED",
   "allowedMissingConnections": [],
   "allowedMissingCodeEnvs": []
}

Summary:
Our original managed folder is getting replaced by folder of project, where the plugin is being used but we are not able to use that folder in original recipe.

> I hope there is a way to dynamically set the managed folder in custom recipes so that it could be taken up directly in custom recipes of our plugin.

https://update.dataiku.com/dss/11/plugins/list.json

The Plugins URL is version specific so for v12 it would use 12 in the URL. It doesn't work on v10 on below so I suspect this is a new URL used by v11 and above only. The URL got me a nice JSON which with a little bit of Python I produced two datasets: plugins and plugin_releases.

I was wondering how to share this project with others but I just realised it will much easier for me to share the Python recipe which will be here for ever and will not depend on any file sharing tools so here it goes:

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
from IPython.display import display, HTML
display(HTML("<style>.container { width:100% !important; }</style>"))

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
import dataiku
from dataiku import pandasutils as pdu
import pandas as pd
import requests, json
from sys import platform

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
# Specify your CA bundle if your OS supports it
if platform == "linux" or platform == "linux2":
    # linux  
    os.environ["REQUESTS_CA_BUNDLE"] = '/etc/ssl/certs/ca-bundle.crt'
elif platform == "darwin":
    # OS X
    print("Use pip to install certifi")
elif platform == "win32":
    # Windows
    print("Use pip to install certifi")

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
client = dataiku.api_client()
dataiku_version = client.get_instance_info().raw['dssVersion'].split(".")[0]

if int(dataiku_version) < 11:
    raise Exception('This only works for Dataiku v11 and above')

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
dataiku_plugins_url = f'https://update.dataiku.com/dss/{dataiku_version}/plugins/list.json'
headers = {'Content-Type': 'application/json'}

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
status_code = 0
status_reason = ''
status_text = ''
    
try:
    response = requests.get(dataiku_plugins_url, headers=headers, verify=True, timeout=(1, 3))
    
    status_code = response.status_code
    status_reason = response.reason
    status_text = str(status_code) + ' - ' + str(status_reason)

    # Raise an exception if the response status code is not successful
    response.raise_for_status()
    
except requests.exceptions.BaseHTTPError:
    error_text = "Base HTTP Error: " + status_text
except requests.exceptions.HTTPError:
    error_text = "HTTP Error: " + status_text
except requests.exceptions.Timeout:
    error_text = "The request timed out"
except requests.exceptions.ConnectionError:
    error_text = "Connection Error"
except requests.exceptions.RequestException:
    error_text = "Unknown error occurred: " + str(status_code)    
    
# If the request was successful
if status_code == 200:
    # Parse the response as JSON
    json_object = response.json()
    
    # Debug: Print the whole JSON object
    # print(json.dumps(json_object, indent=4))

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
df_plugins = pd.DataFrame(columns=['ID', 'Label', 'Description', 'Author', 'Icon', 'Size', 'Store_Version', 'URL', 'Download_URL', 'Support_Level', 'License_Info', 'Downloadable', 'tutorials', 'sampleProjects', 'javaPreparationProcessors', 'javaFormulaFunctions', 'customDatasets', 
                                   'customCodeRecipes', 'customPythonProbes', 'customPythonChecks', 'customSQLProbes', 'customFormats', 'customExporters', 'customPythonSteps', 'customPythonTriggers', 'customRunnables', 'customWebApps', 'customFSProviders', 'customDialects', 
                                   'customJythonProcessors', 'customPythonClusters', 'customParameterSets', 'customFields', 'customJavaPolicyHooks', 'customWebAppExpositions', 'customPythonPredictionAlgos', 'customStandardWebAppTemplates', 'customBokehWebAppTemplates', 
                                   'customShinyWebAppTemplates', 'customRMarkdownReportTemplates', 'customPreBuiltNotebookTemplates', 'customPythonNotebookTemplates', 'customRNotebookTemplates', 'customScalaNotebookTemplates', 'customPreBuiltDatasetNotebookTemplates', 
                                   'customPythonDatasetNotebookTemplates', 'customRDatasetNotebookTemplates', 'customScalaDatasetNotebookTemplates'])

df_plugin_releases = pd.DataFrame(columns=['ID', 'Label', 'Version', 'Release_Date_Time', 'Release_Notes'])

for item in json_object['items']:

    for release in item['revisions']:
        plugin_release_record = pd.DataFrame.from_dict({'ID': [item['id']], 'Label': [item['meta'].get('label', '')], 'Version': [release.get('version', '')], 'Release_Date_Time': [release.get('releaseTime', '')], 'Release_Notes': [release.get('releaseNotes', '')]})
        df_plugin_releases = pd.concat([df_plugin_releases, plugin_release_record], ignore_index=True, sort=False)
    
    plugin_record = pd.DataFrame.from_dict({'ID': [item['id']], 'Label': [item['meta'].get('label', '')], 'Description': [item['meta'].get('description', '')], 'Author': [item['meta'].get('author', '')], 'Icon': [item['meta'].get('icon', '')], 'Size': [item['size']],
                                         'Store_Version': [item['storeVersion']], 'URL': [item['meta'].get('url', '')], 'Download_URL': [item['downloadURL']], 'Support_Level': [item['meta'].get('supportLevel', '')], 'License_Info': [item['meta'].get('licenseInfo', '')],
                                         'Downloadable': [item['storeFlags'].get('downloadable', '')], 'tutorials': [len(item['content']['tutorials'])], 'sampleProjects': [len(item['content']['sampleProjects'])], 'javaPreparationProcessors': [len(item['content']['javaPreparationProcessors'])],
                                         'javaFormulaFunctions': [len(item['content']['javaFormulaFunctions'])], 'customDatasets': [len(item['content']['customDatasets'])], 'customCodeRecipes': [len(item['content']['customCodeRecipes'])],
                                         'customPythonProbes': [len(item['content']['customPythonProbes'])], 'customPythonChecks': [len(item['content']['customPythonChecks'])], 'customSQLProbes': [len(item['content']['customSQLProbes'])], 'customFormats': [len(item['content']['customFormats'])],
                                         'customExporters': [len(item['content']['customExporters'])], 'customPythonSteps': [len(item['content']['customPythonSteps'])], 'customPythonTriggers': [len(item['content']['customPythonTriggers'])], 'customRunnables': [len(item['content']['customRunnables'])],
                                         'customWebApps': [len(item['content']['customWebApps'])], 'customFSProviders': [len(item['content']['customFSProviders'])], 'customDialects': [len(item['content']['customDialects'])], 'customJythonProcessors': [len(item['content']['customJythonProcessors'])],
                                         'customPythonClusters': [len(item['content']['customPythonClusters'])], 'customParameterSets': [len(item['content']['customParameterSets'])], 'customFields': [len(item['content']['customFields'])],
                                         'customJavaPolicyHooks': [len(item['content']['customJavaPolicyHooks'])], 'customWebAppExpositions': [len(item['content']['customWebAppExpositions'])], 'customPythonPredictionAlgos': [len(item['content']['customPythonPredictionAlgos'])],
                                         'customStandardWebAppTemplates': [len(item['content']['customStandardWebAppTemplates'])], 'customBokehWebAppTemplates': [len(item['content']['customBokehWebAppTemplates'])], 'customShinyWebAppTemplates': [len(item['content']['customShinyWebAppTemplates'])],
                                         'customRMarkdownReportTemplates': [len(item['content']['customRMarkdownReportTemplates'])], 'customPreBuiltNotebookTemplates': [len(item['content']['customPreBuiltNotebookTemplates'])],
                                         'customPythonNotebookTemplates': [len(item['content']['customPythonNotebookTemplates'])], 'customRNotebookTemplates': [len(item['content']['customRNotebookTemplates'])], 'customScalaNotebookTemplates': [len(item['content']['customScalaNotebookTemplates'])],
                                         'customPreBuiltDatasetNotebookTemplates': [len(item['content']['customPreBuiltDatasetNotebookTemplates'])], 'customPythonDatasetNotebookTemplates': [len(item['content']['customPythonDatasetNotebookTemplates'])],
                                         'customRDatasetNotebookTemplates': [len(item['content']['customRDatasetNotebookTemplates'])], 'customScalaDatasetNotebookTemplates': [len(item['content']['customScalaDatasetNotebookTemplates'])]})

    df_plugins = pd.concat([df_plugins, plugin_record], ignore_index=True, sort=False)

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
df_plugin_releases['Release_Date_Time'] = pd.to_datetime(df_plugin_releases['Release_Date_Time'],unit='ms')

# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE


# -------------------------------------------------------------------------------- NOTEBOOK-CELL: CODE
# Recipe outputs
plugins = dataiku.Dataset("plugins")
plugins.write_with_schema(df_plugins)
plugin_releases = dataiku.Dataset("plugin_releases")
plugin_releases.write_with_schema(df_plugin_releases)

So to add this to a project, add a Python recipe, set two outputs as follows: plugins and plugin_releases and click Create Recipe. Run it and you will have the two new datasets populated. Now you have an easy way to explore Dataiku plugins and see when they get changed/released. Obviously the Plugins URL has not been formaly published by Dataiku but considering every DSS v11 and v12 is using this URL I would think it's pretty safe to use, even if unsupported. Also if this project breaks is not the end of the world, we are not trying to predict anything here, it's an information tool.

In our case I think I am going to build a scenario to check for new plugin releases daily or weekly, and then post a notification on a Team's channel so our users and myself get notified when new plugin versions get released.

Hope it helps!

Does anyone know of a way to automatically include the current user ID in SQL table name. I've seen references in the documentation to the variable dssUserLogin so I thought the following might work:

${projectKey}_${dssUserLogin}_TABLENAME

However this variable is not recognized in SQL recipes.

I also tried creating a user property but this didn't work either:

${projectKey}_${userProperty:user_id}_TABLENAME

Note that I am on version 8.02 so maybe one of these might work on version 10?

I realize I could create an instance variable but I can't see how I could set this to the current user when the user logs into DSS. I could set it at the beginning of a flow in a Python recipe but that would be pretty awkward and painful to do for every project.

Any ideas?

Thanks,

Marlan

Operating system used: Linux

Is there a way to add custom checks in the same way as adding metrics to a dataset through the dataset api ?

Thanks

I am trying to integrate the MLFLOW models into DSS, which I have successfully done as per the DSS 10 documentation. I am interested in knowing if we would be able to connect with the ground truth data (lets say from (snowflake DB) for the models that we have imported from mlflow into DSS, so that we would be able to view the performance metrics in the model versions tab. Attaching the screenshot where these tabs are disabled due to the unavailability of the ground truth data.