loading

Module to load production and cross-border flows from ENTSO-E

ecodynelec.preprocessing.loading.adjust_exchanges(Cross, neighbourhood, net_exchange=False, freq='H', sg_data=None, is_verbose=False)

Bring adjustments to the exchange data: add SwissGrid data, fill data, adjust frequency and set exchanges to net.

Parameters:

• Cross (dict) – the Cross-border flow data, as dict of pandas DataFrame

• neighbourhood (list) – list of involved countries, as main countries or neighbours

• net_exchange (bool, default to False) – to consider net cross-border flows

• freq (str, default to ‘H’) – time step

• sg_data (pandas.DataFrame, default to None) – information from Swiss Grid

• is_verbose (bool, default to False) – to display information

Returns:

dict of pandas DataFrame with adjusted cross-border flow data.

Return type:

dict

ecodynelec.preprocessing.loading.adjust_generation(Gen, freq='h', residual_global=False, start=None, end=None, sg_data=None, path_gap=None, enr_prod_ch=None, is_verbose=False)

Function that leads organizes the data adjustment. It sorts finds and sorts missing values, fills it, resample the data and add a residual as global production

Parameters:

• Gen (dict) – dict of dataFrames containing the generation for each country

• freq (str, default to ‘H’) – time step durtion

• residual_global (bool, default to False) – whether to include the residual or not

• start (str or None, default to None) – starting date of the study

• end (str or None, default to None) – ending date of the study

• sg_data (pandas.DataFrame, default to None) – information from Swiss Grid

• path_gap (str or None, default to None) – path to the file containing the information about the nature of the residual refer to parameter.path.gap for more information

• enr_prod_ch (pandas.DataFrame, default to None) – Renewable energy production in Switzerland, as modeled with EcoDynElec-Enr-Model See parameter.ch_enr_model_path for more information

• is_verbose (bool, default to False) – whether to display information or not.

Returns:

dict of pandas DataFrames containing modified Gen dict.

Return type:

dict

ecodynelec.preprocessing.loading.create_net_exchange(Cross)

Adapt the cross-border flow to consider exchanges at each border and time step as net. Net exchange means that electricity can only go from A to B or from B to A, but not in both directions at the same time.

ecodynelec.preprocessing.loading.import_data(ctry, start=None, end=None, freq='H', involved_countries=None, path_gap=None, sg_data=None, enr_prod_ch=None, net_exchange=False, path_gen=None, gen_preprocessed=None, path_imp=None, imp_preprocessed=None, savedir=None, residual_global=False, correct_imp=True, clean_data=True, n_hours=2, days_around=7, limit=0.4, is_verbose=True, progress_bar: ProgressInfo | None = None)

Main function managing the import and pre-treatment of Entso-e production and cross-border flow data.

Parameters:

• ctry (list) – list of countries to include in the computing (list)

• start – starting date, as str or datetime

• end – ending date, as str or datetime

• freq (str, default to ‘H’) – frequency of time steps to consider

• involved_countries (list, default to None) – list of all countries involved, with the countries to include in the computing and their neighbours (to implement the exchanges with ‘Other’ countries)

• path_gap (str or None, default to None) – path to the file containing the information about the nature of the residual refer to parameter.path.gap for more information

• sg_data (pandas.DataFrame, default to None) – information from Swiss Grid

• enr_prod_ch (pandas.DataFrame, default to None) – Wind and solar production in Switzerland, as modeled with EcoDynElec-Enr-Model See Parameter.ch_enr_model_path for more information

• net_exchange (bool, default to False) – to simplify cross-border flows to net after resampling

• path_gen (str, default to None) – directory where raw Entso-e generation files are stored

• path_imp (str, default to None) – directory where raw Entso-e files for cross-border flow data are stored

• gen_preprocessed (str, default to None) – directory where preprocessed Entso-e generation files are stored

• imp_preprocessed (str, default to None) – directory containing the files for preprocessed cross-border flow data

• savedir (str, default to None) – directory to save information

• residual_global (bool, default to False) – to consider the production residual as produced electricity that can be exchanged with neighbour countries

• correct_imp (bool, default to False) – to replace cross-border flow of Entso-e for Swizerland with data from Swiss Grid

• clean_data (bool, default to True) – to enable automatic data cleaning / filling

• n_hours (int, default to 2) – max number of successive missing hours to be considered as occasional event

• days_around (int, default to 7) – number of days after and before a gap to consider to create a ‘typical mean day’

• limit (float, default to 0.4) – max relative length of a gap to fill the data. Longer gaps are filled with zeros.

• is_verbose (bool, default to False) – to display information

• progress_bar (ProgressInfo, default to None) – to display a progress bar

Returns:

pandas DataFrame with all productions and all exchanges from all included countries.

Return type:

pandas.DataFrame

ecodynelec.preprocessing.loading.import_exchanges(ctry, start, end, path_imp=None, path_prep=None, savedir=None, freq='H', n_hours: int = 2, days_around: int = 7, limit: float = 0.4, clean_imports: bool = True, is_verbose=False, progress_bar: ProgressInfo | None = None)

Function to import the cross-border flows. Finds the useful files to load, load the data, group relevant information and adjust time step.

Parameters:

• ctry (list) – countries to incldue in the study (list)

• start – starting date, as str or datetime

• end – ending date, as str or datetime

• path_imp (str, default to None) – directory where raw Entso-e exchange files are stored (str) [secondary path]

• path_prep (str, default to None) – directory where preprocessed Entso-e exchange files are stored (str) [prioritary path] Where preprocessed files are saved if both path_prep and path_imp are passed and different.

• savedir (str, default to None) – directory path to save results (str, default: None)

• freq (str, default to ‘H’) – the frequency to consier

• n_hours (int, default to 2) – max number of successive missing hours to be considered as occasional event

• days_around (int, default to 7) – number of days after and before a gap to consider to create a ‘typical mean day’

• limit (float, default to 0.4) – max relative length of a gap to fill the data. Longer gaps are filled with zeros.

• clean_generation (bool, default to True) – to enable automatic data cleaning / filling

• is_verbose (bool, default to False) – to display information

Returns:

dict of pandas.DataFrame containing cross-border flows.

Return type:

dict

ecodynelec.preprocessing.loading.import_generation(ctry, start, end, path_gen=None, path_prep=None, savedir=None, n_hours: int = 2, days_around: int = 7, limit: float = 0.4, clean_generation: bool = True, is_verbose=False, progress_bar: ProgressInfo | None = None)

Function to import generation data from Entso-e information source.

Parameters:

• ctry (list) – countries to incldue in the study (list)

• start – starting date, as str or datetime

• end – ending date, as str or datetime

• path_gen (str, default to None) – directory where raw Entso-e generation files are stored (str) [secondary path]

• path_prep (str, default to None) – directory where preprocessed Entso-e generation files are stored (str) [prioritary path] Where preprocessed files are saved if both path_prep and path_gen are passed and different.

• savedir (str, default to None) – directory path to save results (str, default: None)

• n_hours (int, default to 2) – max number of successive missing hours to be considered as occasional event

• days_around (int, default to 7) – number of days after and before a gap to consider to create a ‘typical mean day’

• limit (float, default to 0.4) – max relative length of a gap to fill the data. Longer gaps are filled with zeros.

• clean_generation (bool, default to True) – to enable automatic data cleaning / filling

• is_verbose (bool, default to False) – to display information

Returns:

processed generation data per country

Return type:

dict

ecodynelec.preprocessing.loading.resample_data(Data, freq)

Function that turns data from MW to MWh and adapts its frequency. The data is assumed to be in MW, in a table with 15min indexes.

Parameters:

• Data (dict) – dict of DataFrames containing the generation data.

• freq (str) – the frequency (length of time step)

Returns:

dict of pandas DataFrame wiht resampled and converted energy

Return type:

dict

ecodynelec.preprocessing.loading.set_swissGrid(Cross, sg_data)

Function to replace the cross-border flow data of ENTSO-E by the cross-border flow data of SwissGrid. Data passed must be in 15min.

Parameters:

• Cross (dict) – the Cross-border flow data, as dict of pandas DataFrame

• sg_data (pandas.DataFrame) – information from Swiss Grid

Returns:

dict of pandas DataFrame with cross-border flow data for all the countries of the studied area.

Return type:

dict