Python CDB Zugriff

Judi · April 21, 2022, 1:04pm

Guten Tag,

Zur weiteren Analyse eines Modells möchte ich die Daten mit Python aus der Datenbank auslesen. Dabei habe ich mich an das bereitgestellte Beispiel “single_span_girder” gehalten.
Die Dateien “sofistik-daten” und “dll” habe ich abgespeichert und den Pfad zur cdb angepasst.
Da ich die educational version nutze, habe ich auch den Namen der dll zu “sof_cdb_w_edu-70.dll” angepasst.

Beim Ausführen meines Python-Codes kommt leider direkt die Fehlermeldung:

NameError: name 'py_sof_cdb_kexist' is not defined

Auf was lässt dies schließen? Ich vermute, dass beim import der cdb funktionen etwas nicht stimmt, weshabl py_sof_cdb_kexist nicht aufgerufen werden kann.
(Was) muss ich weiteres beachten?

Was mich weiterhin verwundert ist, dass der ausgegebene Path sehr lang ist:

C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\MATLAB\R2022a\runtime\win64;C:\Program Files\MATLAB\R2022a\bin;C:\Program Files\MATLAB\R2020a\runtime\win64;C:\Program Files\MATLAB\R2020a\bin;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Users\user\AppData\Local\Programs\Python\Python39\Scripts\;C:\Users\user\AppData\Local\Programs\Python\Python39\;C:\Users\user\AppData\Local\Microsoft\WindowsApps;

Das ist aber vermutlich eher eine Python-Frage. Der Pfad zu Matlab taucht auf, da ich später die Daten in Matlab weiterverarbeite und deshalb den Python code dort ausführen werde.

Über Hilfestellung oder Anmerkungen bin ich sehr Dankbar, da als Erläuterung leider nur die installierten Dateien zur Verfügung stehen.

VG
JuDi

sfr · April 21, 2022, 3:02pm

Seems like you are missing the necessary import statements in the example.
The function py_sof_cdb_kexist is defined in dlls.py

from sofistik_daten import *
import os             # for the environment variable necessary
import platform       # checks the python platform
from dlls import *
from ctypes import *  # read the functions from the cdb

You should start off by trying to run the dlls.py script only, that way you can see if the cdb/dll access is working.

The path should always be long, since scripts/programs need to be able to access:

All the basic window functions (e.g. system32)
Local user settings/functions (e.g. appdata)
Functions specific to the loaded programs python/matlab
The path returned by the error message shows, where python has looked for files/functions

Judi · April 22, 2022, 7:24am

Thanks for the quick reply and explanation!

When I run the dll.py (here connect_to_cdb.py) the message is:

...
CDB Status: 1
CDB closed successfully, status = 0

So I guess it works. Running the following skript:

# +============================================================================+
# | Company:   SOFiSTiK AG                                                     |
# | Version:   SOFiSTIK 2020                                                   |
# +============================================================================+

# import all types from sofistik_daten.py, original file can be found by following
# --> examples/python/sofistik_daten.py
from sofistik_daten import *

import os             # for the environment variable necessary
import platform       # checks the python platform
from connect_to_cdb import *
from ctypes import *  # read the functions from the cdb

# This example has been tested with Python 3.7 (64-bit)
print ("CDB Status:", cdbStat.value)

########################################################### new:
# CDB auslesen
"""
do while ie == 0, see cdbase.chm, Returnvalue.
    = 0 -> No error
    = 1 -> Item is longer than Data
    = 2 -> End of file reached
    = 3 -> Key does not exist
"""

# Zeitpunkt [sec]
if py_sof_cdb_kexist(12, 1000) == 2: # the key exists and contains data 12:global load 1000: LC (ggf. ändern oder variabel gestalten)
    ie = c_int(0)
    rpar = 0.0
    RecLen = c_int(sizeof(clc_ctrl))
    while ie.value < 2:
        ie.value = py_sof_cdb_get(Index, 12, 1000, byref(clc_ctrl), byref(RecLen), 1)
        rpar = clc_ctrl.m_rpar
        RecLen = c_int(sizeof(clc_ctrl))

The message is:

...
CDB Status: 1
CDB closed successfully, status = 0
CDB Status: 0
Traceback (most recent call last):
  File "C:/Users/user/Desktop/D/python/Zugriff cdb/py_zugriff_cdb.py", line 30, in <module>
    if py_sof_cdb_kexist(12, 1000) == 2: # the key exists and contains data 12:global load 1000: LC (ggf. ändern oder variabel gestalten)
NameError: name 'py_sof_cdb_kexist' is not defined

So there must be another error. Since the status is 0, could it be because the cdb is closed?

sfr · April 24, 2022, 5:11am

The error message seems to suggest that the function py_sof_cdb_kexist isn’t defined properly.

I suggest you start from the beginning and work your way up:

Start with the dll.py and rename it (use it as your main file)
include from sofistik_daten import * at the top of the file
Rinse it from the stuff you don’t need
I.e. get rid of the if/else for 32/64-bit architecture and whatever else you find distracting (checking python platform etc)
Start copy pasting code from single_span_girder.py
Start reading keys that you know work/exist
Maybe check the concrete material first (as per example)
Does LC 1000 exist in the cdb you are checking?

Judi · April 25, 2022, 10:13am

Thanks for the support!
Apparently the path to the file sof_cdb_w_edu-70.dll is not found correctly. I added the full path for getting “myDLL” but still it gets the error message:

FileNotFoundError: Could not find module 'C:\Program Files\SOFiSTiK0\SOFiSTiK 2020\interfaces4bit\sof_cdb_w_edu-70.dll' (or one of its dependencies). Try using the full path with constructor syntax.

This is the code I am testing right now:

# +============================================================================+
# | Company:   SOFiSTiK AG                                                     |
# | Version:   SOFiSTIK 2020                                                   |
# +============================================================================+
from sofistik_daten import *
import os             # for the environment variable necessary
import platform       # checks the python platform
import string
from ctypes import *  # read the functions from the cdb

# This example has been tested with Python 3.7 (64-bit)
print ("The path variable=", os.environ["Path"])

# Check the python platform (32bit or 64bit)
print ("Python architecture=", platform.architecture())
sofPlatform = str(platform.architecture())

# Get the dlls ( 64bit DLL)

path = os.environ["Path"]

    # 64bit DLLs
dllPath = r"C:\Program Files\SOFiSTiK\2020\SOFiSTiK 2020\interfaces\64bit"
dllPath += ";"

    # Other necessary DLLs
#dllPath += r"C:\Program Files\SOFiSTiK\2020\SOFiSTiK 2020"
#os.environ["Path"] = dllPath + ";" + path

    # Get the DLL functions
myDLL = cdll.LoadLibrary("C:\Program Files\SOFiSTiK\2020\SOFiSTiK 2020\interfaces\64bit\sof_cdb_w_edu-70.dll")
py_sof_cdb_get = cdll.LoadLibrary("sof_cdb_w_edu-70.dll").sof_cdb_get
py_sof_cdb_get.restype = c_int

py_sof_cdb_kenq = cdll.LoadLibrary("sof_cdb_w_edu-70.dll").sof_cdb_kenq_ex
py_sof_cdb_kexist = cdll.LoadLibrary("sof_cdb_w_edu-70.dll").sof_cdb_kexist

# Connect to CDB
Index = c_int()
cdbIndex = 99

# Set the CDB Path
fileName = r"C:\Users\...\modell 3 mit dummy platten___.cdb"

# important: Unicode call!
Index.value = myDLL.sof_cdb_init(fileName.encode('utf-8'), cdbIndex)

cdbStat = c_int()  # get the CDB status
cdbStat.value = myDLL.sof_cdb_status(Index.value)

# Print the Status of the CDB
cdbStat.value = myDLL.sof_cdb_status(Index.value)
if cdbStat.value == 0:
    print ("CDB closed successfully, status = 0")

The load case 10000 (not 1000 but I changed it) is saved in the database, so this should not be the problem.

Thanks a lot for the help! Now the question is, how can I fix it? Do you have an idea, why the path is not found?

Judi · April 25, 2022, 10:30am

In fact, the exact dll.py code is slightly different from the one I used in connect_to_cdb.py:

# +============================================================================+
# | Company:   SOFiSTiK AG                                                     |
# | Version:   SOFiSTIK 2020                                                   |
# +============================================================================+

import os               # for the environment variable necessary, this is a great tool
import platform         # checks the python platform
import string
from ctypes import *    # read the functions from the cdb

# This example has been tested with Python 3.7 (64-bit)
print ("The path variable=", os.environ["Path"])

# Check the python platform (32bit or 64bit)
print ("Python architecture=", platform.architecture())
sofPlatform = str(platform.architecture())


    # Set environment variable for the dll files
print ("Hint: 64bit DLLs are used")

    # Set DLL dir path - new in PY 3.8 for ctypes
    # See: https://docs.python.org/3/whatsnew/3.8.html#ctypes
os.add_dll_directory(r"C:\Program Files\SOFiSTiK\2020\SOFiSTiK 2020\interfaces\64bit")
os.add_dll_directory(r"C:\Program Files\SOFiSTiK\2020\SOFiSTiK 2020")

    # Get the DLL functions
myDLL = cdll.LoadLibrary("sof_cdb_w_edu-70.dll")
py_sof_cdb_get = cdll.LoadLibrary("sof_cdb_w_edu-70.dll").sof_cdb_get
py_sof_cdb_get.restype = c_int

py_sof_cdb_kenq = cdll.LoadLibrary("sof_cdb_w_edu-70.dll").sof_cdb_kenq_ex

# Connect to CDB
Index = c_int()
cdbIndex = 99

# input the cdb path here
fileName = r"C:\Users\...\modell 3 mit dummy platten___.cdb"

# important: Unicode call!
Index.value = myDLL.sof_cdb_init(fileName.encode('utf8'), cdbIndex)

# get the CDB status
cdbStat = c_int()
cdbStat.value = myDLL.sof_cdb_status(Index.value)

# Print the Status of the CDB
print ("CDB Status:", cdbStat.value)

# Close the CDB, 0 - will close all the files
#myDLL.sof_cdb_close(0)

# Print again the status of the CDB, if status = 0 -> CDB Closed successfully
cdbStat.value = myDLL.sof_cdb_status(Index.value)
if cdbStat.value == 0:
    print ("CDB closed successfully, status = 0")

This one seems to work.

But when I test the database access (now with single_span_girder.cdb and a code from the example) the error message still comes up:

NameError: name 'py_sof_cdb_kexist' is not defined

sfr · April 25, 2022, 11:29am

In the last one you are only defining py_sof_cdb_kenq and missed the row after with py_sof_cdb_kexist

Judi · April 25, 2022, 12:50pm

You got it!
That line should be added in the example code here as well: Connect to CDB — CDB Interfaces 2020

Okay, when I add this code:

# Read the fck value from the CDB
print(py_sof_cdb_kexist(1, 1))
if py_sof_cdb_kexist(1, 1) == 2: # the key exists and contains data
    fck = 0.0
    ie = c_int(0)
    RecLen = c_int(sizeof(cmat_conc))
    while ie.value < 2:
        ie.value = py_sof_cdb_get(Index, 1, 1, byref(cmat_conc), byref(RecLen), 1)
        if cmat_conc.m_id == 1.0:
            fck = cmat_conc.m_fck
        RecLen = c_int(sizeof(cmat_conc))

print(fck)

the next error message is:

NameError: name 'fck' is not defined

The first if-condition is 0, although the key should exist, as it is saved in the database:

sfr · April 25, 2022, 3:59pm

Try changing the if line to: cmat_conc.m_id == 1 (Wild guess that the function returns an int)

Otherwise try: print(cmat_conc.m_id) before the if and see if you are actually accessing the records.

Judi · April 26, 2022, 9:24am

The output is actually 0 for cmat_conc.m_id, so access to the database does not work.

Judi · April 26, 2022, 9:40am

I checked print(Index.value) the output is -48 so there’s an error in the access. In the CDBase-Help is written for cdb_init():

Returnvalue: Index for further access
<0 if error, see clib1.h for meaning of error codes
0 if not a database when called with index = 99

where can I find the error information/clib1.h?

sfr · April 26, 2022, 10:17am

Regarding material id:
I think you are returning the value for the first line in your CDB (1 MAT), that is what the while loop is for.
Once it reads the second row the id should be 1 cmat_conc.m_id == 1
So look at all the printouts of print(cmat_conc.m_id) , not just the first one.
Regarding clib1.h:
I don’t think that one exists (anymore). Error codes are listed here:
Connect to CDB — CDB Interfaces 2020
Seems like you haven’t been able to connect.
Double check whether the cdb is in your working directory for the python script.

Judi · April 26, 2022, 11:11am

Thanks for all the support!

also for print(cmat_conc.m_type) I get the output 0. As I am calling the function cmat_conc and not c_mat I do not think that the wrong data is being accessed, but cannot be accessed at all and thus 0 is being output for everything
I got the single_span_girder.cdb in the working directory and the path for fileName is correct - doesn’t help. The error code for -48 says “file is no CDBase-file”. I tested with oder .cdb files but really don’t understand why it doesn’t work

sfr · April 27, 2022, 11:19am

Are you sure you are in the correct directory in python?
The below should spit out files/folder in the directory:

import os
os.listdir('.')

Judi · April 27, 2022, 11:23am

I found the error. I had to specify “sof_cdb_w-70.dll” without _edu. Now it works. It helped me a lot to understand the code but I really thought I had tested this. Maybe there was another mistake before…