python-weka-wrapper3-0.1.3/0000775017267001726700000000000013147133421017153 5ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/CHANGES.rst0000644017267001726700000000362013147123252020755 0ustar fracpetefracpete00000000000000Changelog ========= 0.1.3 (2017-08-23) ------------------ - added `check_for_modified_class_attribute` method to `FilterClassifier` class - added `complete_classname` method to `weka.core.classes` module, which allows completion of partial classnames like `.J48` to `weka.classifiers.trees.J48` if there is a unique match; `JavaObject.new_instance` and `JavaObject.check_type` now make use of this functionality, allowing for instantiations like `Classifier(cls=".J48")` - `jvm.start(system_cp=True)` no longer fails with a `KeyError: 'CLASSPATH'` if there is no `CLASSPATH` environment variable defined - Libraries `mtl.jar`, `core.jar` and `arpack_combined_all.jar` were added as is to the `weka.jar` in the 3.9.1 release instead of adding their content to it. Repackaged `weka.jar` to fix this issue (https://github.com/fracpete/python-weka-wrapper3/issues/5) 0.1.2 (2017-01-04) ------------------ - `typeconv.double_matrix_to_ndarray` no longer assumes a square matrix (https://github.com/fracpete/python-weka-wrapper3/issues/4) - `len(Instances)` now returns the number of rows in the dataset (module `weka.core.dataset`) - added method `insert_attribute` to the `Instances` class - added class method `create_relational` to the `Attribute` class - upgraded Weka to 3.9.1 0.1.1 (2016-10-19) ------------------ - `plot_learning_curve` method of module `weka.plot.classifiers` now accepts a list of test sets; `*` is index of test set in label template string - added `missing_value()` methods to `weka.core.dataset` module and `Instance` class - output variable `y` for convenience method `create_instances_from_lists` in module `weka.core.dataset` is now optional - added convenience method `create_instances_from_matrices` to `weka.core.dataset` module to easily create an `Instances` object from numpy matrices (x and y) 0.1.0 (2016-05-09) ------------------ - initial release of Python3 port python-weka-wrapper3-0.1.3/MANIFEST.in0000644017267001726700000000026312712217400020705 0ustar fracpetefracpete00000000000000recursive-include python *.arff recursive-include python *.jar recursive-exclude python weka-src.jar recursive-exclude python weka.jar include DESCRIPTION.rst include CHANGES.rst python-weka-wrapper3-0.1.3/PKG-INFO0000664017267001726700000000612313147133421020252 0ustar fracpetefracpete00000000000000Metadata-Version: 1.1 Name: python-weka-wrapper3 Version: 0.1.3 Summary: Python3 wrapper for the Weka Machine Learning Workbench Home-page: https://github.com/fracpete/python-weka-wrapper3 Author: Peter "fracpete" Reutemann Author-email: pythonwekawrapper@gmail.com License: GNU General Public License version 3.0 (GPLv3) Description: The **python-weka-wrapper3** package makes it easy to run Weka algorithms and filters from within Python. It offers access to Weka API using thin wrappers around JNI calls using the **javabridge** package. Changelog ========= 0.1.3 (2017-08-23) ------------------ - added `check_for_modified_class_attribute` method to `FilterClassifier` class - added `complete_classname` method to `weka.core.classes` module, which allows completion of partial classnames like `.J48` to `weka.classifiers.trees.J48` if there is a unique match; `JavaObject.new_instance` and `JavaObject.check_type` now make use of this functionality, allowing for instantiations like `Classifier(cls=".J48")` - `jvm.start(system_cp=True)` no longer fails with a `KeyError: 'CLASSPATH'` if there is no `CLASSPATH` environment variable defined - Libraries `mtl.jar`, `core.jar` and `arpack_combined_all.jar` were added as is to the `weka.jar` in the 3.9.1 release instead of adding their content to it. Repackaged `weka.jar` to fix this issue (https://github.com/fracpete/python-weka-wrapper3/issues/5) 0.1.2 (2017-01-04) ------------------ - `typeconv.double_matrix_to_ndarray` no longer assumes a square matrix (https://github.com/fracpete/python-weka-wrapper3/issues/4) - `len(Instances)` now returns the number of rows in the dataset (module `weka.core.dataset`) - added method `insert_attribute` to the `Instances` class - added class method `create_relational` to the `Attribute` class - upgraded Weka to 3.9.1 0.1.1 (2016-10-19) ------------------ - `plot_learning_curve` method of module `weka.plot.classifiers` now accepts a list of test sets; `*` is index of test set in label template string - added `missing_value()` methods to `weka.core.dataset` module and `Instance` class - output variable `y` for convenience method `create_instances_from_lists` in module `weka.core.dataset` is now optional - added convenience method `create_instances_from_matrices` to `weka.core.dataset` module to easily create an `Instances` object from numpy matrices (x and y) 0.1.0 (2016-05-09) ------------------ - initial release of Python3 port Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: License :: OSI Approved :: GNU General Public License (GPL) Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence Classifier: Programming Language :: Python :: 3 python-weka-wrapper3-0.1.3/DESCRIPTION.rst0000644017267001726700000000031612712221277021472 0ustar fracpetefracpete00000000000000The **python-weka-wrapper3** package makes it easy to run Weka algorithms and filters from within Python. It offers access to Weka API using thin wrappers around JNI calls using the **javabridge** package. python-weka-wrapper3-0.1.3/setup.py0000644017267001726700000000721013147133243020665 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # setup.py # Copyright (C) 2014-2017 Fracpete (pythonwekawrapper at gmail dot com) import os from setuptools import setup from urllib.request import urlopen, URLError, HTTPError def download_file(url, outfile): """ Downloads the file associated with the URL and saves it to the specified output file. Taken from here: http://stackoverflow.com/a/4028894 :param url: the URL to download :type url: str :param outfile: the name of the output file :type outfile: str :returns: whether the download was successful :rtype: bool """ try: # Open the url f = urlopen(url) print("Downloading '" + url + "' to '" + outfile + "'") # Open our local file for writing with open(outfile, "wb") as local_file: local_file.write(f.read()) # handle errors except HTTPError as e: print("HTTP Error: " + str(e.code) + " " + url) return False except URLError as e: print("URL Error: " + str(e.reason) + " " + url) return False return True def download_weka(): """ Downloads the Weka jar from github associated with this release if nececssary. """ url = "https://github.com/fracpete/python-weka-wrapper3/raw/a3f24d02205168ac12de9ec14be3465cd4882006/python/weka/lib/weka.jar" outfile = os.path.join(os.path.dirname(__file__), "python", "weka", "lib", "weka.jar") if not os.path.exists(outfile): if not download_file(url, outfile): print("Failed to download Weka jar '" + url + "' to '" + outfile + "'!") else: print("Download of Weka jar successful!") def ext_modules(): """ Initiates Weka jar download. """ download_weka() def _read(f): """ Reads in the content of the file. :param f: the file to read :type f: str :return: the content :rtype: str """ return open(f, 'rb').read() setup( name="python-weka-wrapper3", description="Python3 wrapper for the Weka Machine Learning Workbench", long_description=( _read('DESCRIPTION.rst') + b'\n' + _read('CHANGES.rst')).decode('utf-8'), url="https://github.com/fracpete/python-weka-wrapper3", classifiers=[ 'Development Status :: 4 - Beta', 'License :: OSI Approved :: GNU General Public License (GPL)', 'Topic :: Scientific/Engineering :: Artificial Intelligence', 'Programming Language :: Python :: 3', ], license='GNU General Public License version 3.0 (GPLv3)', package_dir={ '': 'python' }, packages=[ "weka", "weka.core", "weka.flow", "weka.plot" ], package_data={ "weka": ["lib/*.jar"], }, include_package_data=True, version="0.1.3", author='Peter "fracpete" Reutemann', author_email='pythonwekawrapper@gmail.com', install_requires=[ "javabridge>=1.0.14", "numpy" ], extras_require={ 'plots': ["matplotlib"], 'graphs': ["pygraphviz", "PIL"], }, ext_modules=ext_modules(), ) python-weka-wrapper3-0.1.3/python/0000775017267001726700000000000013147133421020474 5ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/python/python_weka_wrapper3.egg-info/0000775017267001726700000000000013147133421026341 5ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/python/python_weka_wrapper3.egg-info/requires.txt0000644017267001726700000000010613147133420030733 0ustar fracpetefracpete00000000000000javabridge>=1.0.14 numpy [graphs] pygraphviz PIL [plots] matplotlib python-weka-wrapper3-0.1.3/python/python_weka_wrapper3.egg-info/top_level.txt0000644017267001726700000000000513147133420031063 0ustar fracpetefracpete00000000000000weka python-weka-wrapper3-0.1.3/python/python_weka_wrapper3.egg-info/dependency_links.txt0000644017267001726700000000000113147133420032404 0ustar fracpetefracpete00000000000000 python-weka-wrapper3-0.1.3/python/python_weka_wrapper3.egg-info/PKG-INFO0000644017267001726700000000612313147133420027435 0ustar fracpetefracpete00000000000000Metadata-Version: 1.1 Name: python-weka-wrapper3 Version: 0.1.3 Summary: Python3 wrapper for the Weka Machine Learning Workbench Home-page: https://github.com/fracpete/python-weka-wrapper3 Author: Peter "fracpete" Reutemann Author-email: pythonwekawrapper@gmail.com License: GNU General Public License version 3.0 (GPLv3) Description: The **python-weka-wrapper3** package makes it easy to run Weka algorithms and filters from within Python. It offers access to Weka API using thin wrappers around JNI calls using the **javabridge** package. Changelog ========= 0.1.3 (2017-08-23) ------------------ - added `check_for_modified_class_attribute` method to `FilterClassifier` class - added `complete_classname` method to `weka.core.classes` module, which allows completion of partial classnames like `.J48` to `weka.classifiers.trees.J48` if there is a unique match; `JavaObject.new_instance` and `JavaObject.check_type` now make use of this functionality, allowing for instantiations like `Classifier(cls=".J48")` - `jvm.start(system_cp=True)` no longer fails with a `KeyError: 'CLASSPATH'` if there is no `CLASSPATH` environment variable defined - Libraries `mtl.jar`, `core.jar` and `arpack_combined_all.jar` were added as is to the `weka.jar` in the 3.9.1 release instead of adding their content to it. Repackaged `weka.jar` to fix this issue (https://github.com/fracpete/python-weka-wrapper3/issues/5) 0.1.2 (2017-01-04) ------------------ - `typeconv.double_matrix_to_ndarray` no longer assumes a square matrix (https://github.com/fracpete/python-weka-wrapper3/issues/4) - `len(Instances)` now returns the number of rows in the dataset (module `weka.core.dataset`) - added method `insert_attribute` to the `Instances` class - added class method `create_relational` to the `Attribute` class - upgraded Weka to 3.9.1 0.1.1 (2016-10-19) ------------------ - `plot_learning_curve` method of module `weka.plot.classifiers` now accepts a list of test sets; `*` is index of test set in label template string - added `missing_value()` methods to `weka.core.dataset` module and `Instance` class - output variable `y` for convenience method `create_instances_from_lists` in module `weka.core.dataset` is now optional - added convenience method `create_instances_from_matrices` to `weka.core.dataset` module to easily create an `Instances` object from numpy matrices (x and y) 0.1.0 (2016-05-09) ------------------ - initial release of Python3 port Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: License :: OSI Approved :: GNU General Public License (GPL) Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence Classifier: Programming Language :: Python :: 3 python-weka-wrapper3-0.1.3/python/python_weka_wrapper3.egg-info/SOURCES.txt0000644017267001726700000000263113147133421030225 0ustar fracpetefracpete00000000000000CHANGES.rst DESCRIPTION.rst MANIFEST.in setup.cfg setup.py python/python_weka_wrapper3.egg-info/PKG-INFO python/python_weka_wrapper3.egg-info/SOURCES.txt python/python_weka_wrapper3.egg-info/dependency_links.txt python/python_weka_wrapper3.egg-info/requires.txt python/python_weka_wrapper3.egg-info/top_level.txt python/weka/__init__.py python/weka/associations.py python/weka/attribute_selection.py python/weka/classifiers.py python/weka/clusterers.py python/weka/datagenerators.py python/weka/experiments.py python/weka/filters.py python/weka/core/__init__.py python/weka/core/capabilities.py python/weka/core/classes.py python/weka/core/converters.py python/weka/core/database.py python/weka/core/dataset.py python/weka/core/jvm.py python/weka/core/packages.py python/weka/core/serialization.py python/weka/core/stemmers.py python/weka/core/stopwords.py python/weka/core/tokenizers.py python/weka/core/typeconv.py python/weka/core/version.py python/weka/flow/__init__.py python/weka/flow/base.py python/weka/flow/container.py python/weka/flow/control.py python/weka/flow/conversion.py python/weka/flow/sink.py python/weka/flow/source.py python/weka/flow/transformer.py python/weka/lib/python-weka-wrapper-src.jar python/weka/lib/python-weka-wrapper.jar python/weka/plot/__init__.py python/weka/plot/classifiers.py python/weka/plot/clusterers.py python/weka/plot/dataset.py python/weka/plot/experiments.py python/weka/plot/graph.pypython-weka-wrapper3-0.1.3/python/weka/0000775017267001726700000000000013147133421021423 5ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/python/weka/filters.py0000644017267001726700000003102112712225053023440 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # filters.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import os import sys import argparse import weka.core.jvm as jvm from weka.core.classes import OptionHandler, join_options from weka.core.capabilities import Capabilities from weka.core.converters import Loader from weka.core.converters import Saver from weka.core.dataset import Instances from weka.core.dataset import Instance from weka.core.stemmers import Stemmer from weka.core.stopwords import Stopwords from weka.core.tokenizers import Tokenizer # logging setup logger = logging.getLogger("weka.filters") class Filter(OptionHandler): """ Wrapper class for filters. """ def __init__(self, classname="weka.filters.AllFilter", jobject=None, options=None): """ Initializes the specified filter using either the classname or the supplied JB_Object. :param classname: the classname of the filter :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Filter.new_instance(classname) self.enforce_type(jobject, "weka.filters.Filter") super(Filter, self).__init__(jobject=jobject, options=options) self.__input = javabridge.make_call(self.jobject, "input", "(Lweka/core/Instance;)Z") self.__batchfinished = javabridge.make_call(self.jobject, "batchFinished", "()Z") self.__output = javabridge.make_call(self.jobject, "output", "()Lweka/core/Instance;") self.__outputformat = javabridge.make_call(self.jobject, "getOutputFormat", "()Lweka/core/Instances;") def capabilities(self): """ Returns the capabilities of the filter. :return: the capabilities :rtype: Capabilities """ return Capabilities(javabridge.call(self.jobject, "getCapabilities", "()Lweka/core/Capabilities;")) def inputformat(self, data): """ Sets the input format. :param data: the data to use as input :type data: Instances """ return javabridge.call(self.jobject, "setInputFormat", "(Lweka/core/Instances;)Z", data.jobject) def input(self, inst): """ Inputs the Instance. :param inst: the instance to filter :type inst: Instance :return: True if filtered can be collected from output :rtype: bool """ return self.__input(inst.jobject) def batch_finished(self): """ Signals the filter that the batch of data has finished. :return: True if instances can be collected from the output :rtype: bool """ return self.__batchfinished() def outputformat(self): """ Returns the output format. :return: the output format :rtype: Instances """ inst = self.__outputformat() if inst is None: return None else: return Instances(inst) def output(self): """ Outputs the filtered Instance. :return: the filtered instance :rtype: an Instance object """ return Instance(jobject=self.__output()) def filter(self, data): """ Filters the dataset(s). When providing a list, this can be used to create compatible train/test sets, since the filter only gets initialized with the first dataset and all subsequent datasets get transformed using the same setup. NB: inputformat(Instances) must have been called beforehand. :param data: the Instances to filter :type data: Instances or list of Instances :return: the filtered Instances object(s) :rtype: Instances or list of Instances """ if isinstance(data, list): result = [] for d in data: result.append(Instances(javabridge.static_call( "Lweka/filters/Filter;", "useFilter", "(Lweka/core/Instances;Lweka/filters/Filter;)Lweka/core/Instances;", d.jobject, self.jobject))) return result else: return Instances(javabridge.static_call( "Lweka/filters/Filter;", "useFilter", "(Lweka/core/Instances;Lweka/filters/Filter;)Lweka/core/Instances;", data.jobject, self.jobject)) @classmethod def make_copy(cls, flter): """ Creates a copy of the filter. :param flter: the filter to copy :type flter: Filter :return: the copy of the filter :rtype: Filter """ return Filter( jobject=javabridge.static_call( "weka/filters/Filter", "makeCopy", "(Lweka/filters/Filter;)Lweka/filters/Filter;", flter.jobject)) class MultiFilter(Filter): """ Wrapper class for weka.filters.MultiFilter. """ def __init__(self, jobject=None, options=None): """ Initializes the MultiFilter instance using either creating new instance or using the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: list of commandline options :type options: list """ if jobject is None: classname = "weka.filters.MultiFilter" jobject = MultiFilter.new_instance(classname) self.enforce_type(jobject, "weka.filters.MultiFilter") super(MultiFilter, self).__init__(jobject=jobject, options=options) @property def filters(self): """ Returns the list of base filters. :return: the filter list :rtype: list """ objects = javabridge.get_env().get_object_array_elements( javabridge.call(self.jobject, "getFilters", "()[Lweka/filters/Filter;")) result = [] for obj in objects: result.append(Filter(jobject=obj)) return result @filters.setter def filters(self, filters): """ Sets the base filters. :param filters: the list of base filters to use :type filters: list """ obj = [] for fltr in filters: obj.append(fltr.jobject) javabridge.call(self.jobject, "setFilters", "([Lweka/filters/Filter;)V", obj) class StringToWordVector(Filter): """ Wrapper class for weka.filters.unsupervised.attribute.StringToWordVector. """ def __init__(self, jobject=None, options=None): """ Initializes the StringToWordVector instance using either creating new instance or using the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: list of commandline options :type options: list """ if jobject is None: classname = "weka.filters.unsupervised.attribute.StringToWordVector" jobject = StringToWordVector.new_instance(classname) self.enforce_type(jobject, "weka.filters.unsupervised.attribute.StringToWordVector") super(StringToWordVector, self).__init__(jobject=jobject, options=options) @property def stemmer(self): """ Returns the stemmer. :return: the stemmer :rtype: Stemmer """ return Stemmer( jobject=javabridge.call(self.jobject, "getStemmer", "()Lweka/core/stemmers/Stemmer;")) @stemmer.setter def stemmer(self, stemmer): """ Sets the stemmer. :param stemmer: the stemmer to use :type stemmer: Stemmer """ javabridge.call( self.jobject, "setStemmer", "(Lweka/core/stemmers/Stemmer;)V", stemmer.jobject) @property def stopwords(self): """ Returns the stopwords handler. :return: the stopwords handler :rtype: Stopwords """ return Stopwords( jobject=javabridge.call(self.jobject, "getStopwordsHandler", "()Lweka/core/stopwords/StopwordsHandler;")) @stopwords.setter def stopwords(self, stopwords): """ Sets the stopwords handler. :param stopwords: the stopwords handler to use :type stopwords: Stopwords """ javabridge.call( self.jobject, "setStopwordsHandler", "(Lweka/core/stopwords/StopwordsHandler;)V", stopwords.jobject) @property def tokenizer(self): """ Returns the tokenizer. :return: the tokenizer :rtype: Tokenizer """ return Tokenizer( jobject=javabridge.call(self.jobject, "getTokenizer", "()Lweka/core/tokenizers/Tokenizer;")) @tokenizer.setter def tokenizer(self, tokenizer): """ Sets the tokenizer. :param tokenizer: the tokenizer to use :type tokenizer: Tokenizer """ javabridge.call( self.jobject, "setTokenizer", "(Lweka/core/tokenizers/Tokenizer;)V", tokenizer.jobject) def main(): """ Runs a filter from the command-line. Calls JVM start/stop automatically. Use -h to see all options. """ parser = argparse.ArgumentParser( description='Executes a filter from the command-line. Calls JVM start/stop automatically.') parser.add_argument("-j", metavar="classpath", dest="classpath", help="additional classpath, jars/directories") parser.add_argument("-X", metavar="heap", dest="heap", help="max heap size for jvm, e.g., 512m") parser.add_argument("-i", metavar="input1", dest="input1", required=True, help="input file 1") parser.add_argument("-o", metavar="output1", dest="output1", required=True, help="output file 1") parser.add_argument("-r", metavar="input2", dest="input2", help="input file 2") parser.add_argument("-s", metavar="output2", dest="output2", help="output file 2") parser.add_argument("-c", metavar="classindex", default="-1", dest="classindex", help="1-based class attribute index") parser.add_argument("filter", help="filter classname, e.g., weka.filters.AllFilter") parser.add_argument("option", nargs=argparse.REMAINDER, help="additional filter options") parsed = parser.parse_args() if parsed.input2 is None and parsed.output2 is not None: raise Exception("No second input file provided ('-r ...')!") jars = [] if parsed.classpath is not None: jars = parsed.classpath.split(os.pathsep) params = [] if parsed.input1 is not None: params.extend(["-i", parsed.input1]) if parsed.output1 is not None: params.extend(["-o", parsed.output1]) if parsed.input2 is not None: params.extend(["-r", parsed.input2]) if parsed.output2 is not None: params.extend(["-s", parsed.output2]) if parsed.classindex is not None: params.extend(["-c", parsed.classindex]) jvm.start(jars, max_heap_size=parsed.heap, packages=True) logger.debug("Commandline: " + join_options(sys.argv[1:])) try: flter = Filter(parsed.filter) if len(parsed.option) > 0: flter.options = parsed.option loader = Loader(classname="weka.core.converters.ArffLoader") in1 = loader.load_file(parsed.input1) cls = parsed.classindex if str(parsed.classindex) == "first": cls = "0" if str(parsed.classindex) == "last": cls = str(in1.num_attributes - 1) in1.class_index = int(cls) flter.inputformat(in1) out1 = flter.filter(in1) saver = Saver(classname="weka.core.converters.ArffSaver") saver.save_file(out1, parsed.output1) if parsed.input2 is not None: in2 = loader.load_file(parsed.input2) in2.class_index = int(cls) out2 = flter.filter(in2) saver.save_file(out2, parsed.output2) except Exception as e: print(e) finally: jvm.stop() if __name__ == "__main__": try: main() except Exception as ex: print(ex) python-weka-wrapper3-0.1.3/python/weka/core/0000775017267001726700000000000013147133421022353 5ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/python/weka/core/version.py0000644017267001726700000000173212712225120024406 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # version.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) from weka.core.classes import JavaObject def weka_version(): """ Determines the version of Weka in use. :return: the version :rtype: str """ jobj = JavaObject(JavaObject.new_instance(classname="weka.core.Version")) return str(jobj) python-weka-wrapper3-0.1.3/python/weka/core/stopwords.py0000644017267001726700000000371212712225107024772 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # stopwords.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge from weka.core.classes import OptionHandler class Stopwords(OptionHandler): """ Wrapper class for stopwords handlers. """ def __init__(self, classname="weka.core.stopwords.Null", jobject=None, options=None): """ Initializes the specified stopwords handler using either the classname or the supplied JB_Object. :param classname: the classname of the stopwords handler :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Stopwords.new_instance(classname) self.enforce_type(jobject, "weka.core.stopwords.StopwordsHandler") super(Stopwords, self).__init__(jobject=jobject, options=options) self.__is_stopword = javabridge.make_call(self.jobject, "isStopword", "(Ljava/lang/String;)Z") def is_stopword(self, s): """ Checks a string whether it is a stopword. :param s: the string to check :type s: str :return: True if a stopword :rtype: bool """ return self.__is_stopword(javabridge.get_env().new_string_utf(s)) python-weka-wrapper3-0.1.3/python/weka/core/serialization.py0000644017267001726700000000774712712225101025611 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # serialization.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import weka.core.classes as classes from weka.core.classes import JavaObject from javabridge.jutil import JavaException # logging setup logger = logging.getLogger(__name__) def deepcopy(obj): """ Creates a deep copy of the JavaObject (or derived class) or JB_Object. :param obj: the object to create a copy of :type obj: object :return: the copy, None if failed to copy :rtype: object """ if isinstance(obj, JavaObject): wrapped = True jobject = obj.jobject else: wrapped = False jobject = obj try: serialized = javabridge.make_instance("weka/core/SerializedObject", "(Ljava/lang/Object;)V", jobject) jcopy = javabridge.call(serialized, "getObject", "()Ljava/lang/Object;") if wrapped: jcopy = obj.__class__(jobject=jcopy) return jcopy except JavaException as e: print("Failed to create copy of " + classes.get_classname(obj) + ": " + str(e)) return None def read(filename): """ Reads the serialized object from disk. Caller must wrap object in appropriate Python wrapper class. :param filename: the file with the serialized object :type filename: str :return: the JB_Object :rtype: JB_Object """ return javabridge.static_call( "Lweka/core/SerializationHelper;", "read", "(Ljava/lang/String;)Ljava/lang/Object;", filename) def read_all(filename): """ Reads the serialized objects from disk. Caller must wrap objects in appropriate Python wrapper classes. :param filename: the file with the serialized objects :type filename: str :return: the list of JB_OBjects :rtype: list """ array = javabridge.static_call( "Lweka/core/SerializationHelper;", "readAll", "(Ljava/lang/String;)[Ljava/lang/Object;", filename) if array is None: return None else: return javabridge.get_env().get_object_array_elements(array) def write(filename, jobject): """ Serializes the object to disk. JavaObject instances get automatically unwrapped. :param filename: the file to serialize the object to :type filename: str :param jobject: the object to serialize :type jobject: JB_Object or JavaObject """ if isinstance(jobject, JavaObject): jobject = jobject.jobject javabridge.static_call( "Lweka/core/SerializationHelper;", "write", "(Ljava/lang/String;Ljava/lang/Object;)V", filename, jobject) def write_all(filename, jobjects): """ Serializes the list of objects to disk. JavaObject instances get automatically unwrapped. :param filename: the file to serialize the object to :type filename: str :param jobjects: the list of objects to serialize :type jobjects: list """ array = javabridge.get_env().make_object_array(len(jobjects), javabridge.get_env().find_class("java/lang/Object")) for i in range(len(jobjects)): obj = jobjects[i] if isinstance(obj, JavaObject): obj = obj.jobject javabridge.get_env().set_object_array_element(array, i, obj) javabridge.static_call( "Lweka/core/SerializationHelper;", "writeAll", "(Ljava/lang/String;[Ljava/lang/Object;)V", filename, array) python-weka-wrapper3-0.1.3/python/weka/core/dataset.py0000644017267001726700000013474313003207020024350 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # dataset.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import numpy from weka.core.classes import JavaObject import weka.core.typeconv as typeconv # logging setup logger = logging.getLogger(__name__) class Instances(JavaObject): """ Wrapper class for weka.core.Instances. """ def __init__(self, jobject): """ Initializes the weka.core.Instances wrapper. :param jobject: the weka.core.Instances object to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.core.Instances") super(Instances, self).__init__(jobject) self.__attribute = javabridge.make_call(self.jobject, "attribute", "(I)Lweka/core/Attribute;") self.__attribute_by_name = javabridge.make_call(self.jobject, "attribute", "(Ljava/lang/String;)Lweka/core/Attribute;") self.__num_attributes = javabridge.make_call(self.jobject, "numAttributes", "()I") self.__num_instances = javabridge.make_call(self.jobject, "numInstances", "()I") self.__get_class_index = javabridge.make_call(self.jobject, "classIndex", "()I") self.__set_class_index = javabridge.make_call(self.jobject, "setClassIndex", "(I)V") self.__class_attribute = javabridge.make_call(self.jobject, "classAttribute", "()Lweka/core/Attribute;") self.__get_instance = javabridge.make_call(self.jobject, "instance", "(I)Lweka/core/Instance;") self.__set_instance = javabridge.make_call(self.jobject, "set", "(ILweka/core/Instance;)Lweka/core/Instance;") self.__append_instance = javabridge.make_call(self.jobject, "add", "(Lweka/core/Instance;)Z") self.__insert_instance = javabridge.make_call(self.jobject, "add", "(ILweka/core/Instance;)V") def __iter__(self): """ Allows iterating over the rows. :return: the iterator :rtype: InstanceIterator """ return InstanceIterator(self) def __len__(self): """ Returns the number of rows in the dataset. :return: the number of Instance objects :rtype: int """ return self.num_instances @property def relationname(self): """ Returns the name of the dataset. :return: the name :rtype: str """ return javabridge.call(self.jobject, "relationName", "()Ljava/lang/String;") @relationname.setter def relationname(self, value): """ Sets the name of the dataset. :param value: the name :type value: str """ javabridge.call(self.jobject, "setRelationName", "(Ljava/lang/String;)V", value) @property def num_attributes(self): """ Returns the number of attributes. :return: the number of attributes :rtype: int """ return self.__num_attributes() def attributes(self): """ Returns an iterator over the attributes. """ return AttributeIterator(self) def attribute(self, index): """ Returns the specified attribute. :param index: the 0-based index of the attribute :type index: int :return: the attribute :rtype: Attribute """ return Attribute(self.__attribute(index)) def attribute_by_name(self, name): """ Returns the specified attribute, None if not found. :param name: the name of the attribute :type name: str :return: the attribute or None :rtype: Attribute """ att = self.__attribute_by_name(javabridge.get_env().new_string(name)) if att is None: return None else: return Attribute(att) def attribute_stats(self, index): """ Returns the specified attribute statistics. :param index: the 0-based index of the attribute :type index: int :return: the attribute statistics :rtype: AttributeStats """ return AttributeStats(javabridge.call(self.jobject, "attributeStats", "(I)Lweka/core/AttributeStats;", index)) def values(self, index): """ Returns the internal values of this attribute from all the instance objects. :return: the values as numpy array :rtype: list """ values = [] for i in range(self.num_instances): inst = self.get_instance(i) values.append(inst.get_value(index)) return numpy.array(values) @property def num_instances(self): """ Returns the number of instances. :return: the number of instances :rtype: int """ return self.__num_instances() @property def class_attribute(self): """ Returns the currently set class attribute. :return: the class attribute :rtype: Attribute """ return Attribute(self.__class_attribute()) @property def class_index(self): """ Returns the currently set class index (0-based). :return: the class index, -1 if not set :rtype: int """ return self.__get_class_index() @class_index.setter def class_index(self, index): """ Sets the class index (0-based). :param index: the new index, use -1 to unset :type index: int """ self.__set_class_index(index) def has_class(self): """ Returns whether a class attribute is set (convenience method). :return: whether a class attribute is currently set :rtype: bool """ return self.class_index != -1 def no_class(self): """ Unsets the class attribute (convenience method). """ self.class_index = -1 def class_is_first(self): """ Sets the first attribute as class attribute (convenience method). """ self.class_index = 0 def class_is_last(self): """ Sets the last attribute as class attribute (convenience method). """ self.class_index = self.num_attributes - 1 def get_instance(self, index): """ Returns the Instance object at the specified location. :param index: the 0-based index of the instance :type index: int :return: the instance :rtype: Instance """ return Instance(self.__get_instance(index)) def add_instance(self, inst, index=None): """ Adds the specified instance to the dataset. :param inst: the Instance to add :type inst: Instance :param index: the 0-based index where to add the Instance :type index: int """ if index is None: self.__append_instance(inst.jobject) else: self.__insert_instance(index, inst.jobject) def set_instance(self, index, inst): """ Sets the Instance at the specified location in the dataset. :param index: the 0-based index of the instance to replace :type index: int :param inst: the Instance to set :type inst: Instance :return: the instance :rtype: Instance """ return Instance( self.__set_instance(index, inst.jobject)) def delete(self, index=None): """ Removes either the specified Instance or all Instance objects. :param index: the 0-based index of the instance to remove :type index: int """ if index is None: javabridge.call(self.jobject, "delete", "()V") else: javabridge.call(self.jobject, "delete", "(I)V", index) def delete_attribute(self, index): """ Deletes an attribute at the given position. :param index: the 0-based index of the attribute to remove :type index: int """ javabridge.call(self.jobject, "deleteAttributeAt", "(I)V", index) def delete_first_attribute(self): """ Deletes the first attribute. """ self.delete_attribute(0) def delete_last_attribute(self): """ Deletes the last attribute. """ self.delete_attribute(self.num_attributes - 1) def delete_attribute_type(self, typ): """ Deletes all attributes of the given type in the dataset. :param typ: the attribute type to remove, see weka.core.Attribute Javadoc :type typ: int """ javabridge.call(self.jobject, "deleteAttributeType", "(I)V", typ) def delete_with_missing(self, index): """ Deletes all rows that have a missing value at the specified attribute index. :param index: the attribute index to check for missing attributes :type index: int """ javabridge.call(self.jobject, "deleteWithMissing", "(I)V", index) def insert_attribute(self, att, index): """ Inserts the attribute at the specified location. :param att: the attribute to insert :type att: Attribute :param index: the index to insert the attribute at :type index: int """ javabridge.call(self.jobject, "insertAttributeAt", "(Lweka/core/Attribute;I)V", att.jobject, index) def compactify(self): """ Compactifies the set of instances. """ javabridge.call(self.jobject, "compactify", "()V") def sort(self, index): """ Sorts the dataset using the specified attribute index. :param index: the index of the attribute :type index: int """ javabridge.call(self.jobject, "sort", "(I)V", index) def randomize(self, random): """ Randomizes the dataset using the random number generator. :param random: the random number generator to use :type random: Random """ javabridge.call(self.jobject, "randomize", "(Ljava/util/Random;)V", random.jobject) def stratify(self, folds): """ Stratifies the data after randomization for nominal class attributes. :param folds: the number of folds to perform the stratification for :type folds: int """ javabridge.call(self.jobject, "stratify", "(I)V", folds) def train_cv(self, num_folds, fold, random=None): """ Generates a training fold for cross-validation. :param num_folds: the number of folds of cross-validation, eg 10 :type num_folds: int :param fold: the current fold (0-based) :type fold: int :param random: the random number generator :type random: Random :return: the training fold :rtype: Instances """ if random is None: return Instances( javabridge.call(self.jobject, "trainCV", "(II)Lweka/core/Instances;", num_folds, fold)) else: return Instances( javabridge.call(self.jobject, "trainCV", "(IILjava/util/Random;)Lweka/core/Instances;", num_folds, fold, random.jobject)) def test_cv(self, num_folds, fold): """ Generates a test fold for cross-validation. :param num_folds: the number of folds of cross-validation, eg 10 :type num_folds: int :param fold: the current fold (0-based) :type fold: int :return: the training fold :rtype: Instances """ return Instances( javabridge.call(self.jobject, "testCV", "(II)Lweka/core/Instances;", num_folds, fold)) def equal_headers(self, inst): """ Compares this dataset against the given one in terms of attributes. :param inst: the dataset to compare against :type inst: Instances :return: None if the same, otherwise an error message :rtype: str """ return javabridge.call( self.jobject, "equalHeadersMsg", "(Lweka/core/Instances;)Ljava/lang/String;", inst.jobject) @classmethod def copy_instances(cls, dataset, from_row=None, num_rows=None): """ Creates a copy of the Instances. If either from_row or num_rows are None, then all of the data is being copied. :param dataset: the original dataset :type dataset: Instances :param from_row: the 0-based start index of the rows to copy :type from_row: int :param num_rows: the number of rows to copy :type num_rows: int :return: the copy of the data :rtype: Instances """ if from_row is None or num_rows is None: return Instances( javabridge.make_instance( "weka/core/Instances", "(Lweka/core/Instances;)V", dataset.jobject)) else: dataset = cls.copy_instances(dataset) return Instances( javabridge.make_instance( "weka/core/Instances", "(Lweka/core/Instances;II)V", dataset.jobject, from_row, num_rows)) @classmethod def template_instances(cls, dataset, capacity=0): """ Uses the Instances as template to create an empty dataset. :param dataset: the original dataset :type dataset: Instances :param capacity: how many data rows to reserve initially (see compactify) :type capacity: int :return: the empty dataset :rtype: Instances """ return Instances( javabridge.make_instance( "weka/core/Instances", "(Lweka/core/Instances;I)V", dataset.jobject, capacity)) @classmethod def create_instances(cls, name, atts, capacity): """ Creates a new Instances. :param name: the relation name :type name: str :param atts: the list of attributes to use for the dataset :type atts: list of Attribute :param capacity: how many data rows to reserve initially (see compactify) :type capacity: int :return: the dataset :rtype: Instances """ attributes = [] for att in atts: attributes.append(att.jobject) return Instances( javabridge.make_instance( "weka/core/Instances", "(Ljava/lang/String;Ljava/util/ArrayList;I)V", name, javabridge.make_list(attributes), capacity)) @classmethod def merge_instances(cls, inst1, inst2): """ Merges the two datasets (side-by-side). :param inst1: the first dataset :type inst1: Instances or str :param inst2: the first dataset :type inst2: Instances :return: the combined dataset :rtype: Instances """ return Instances(javabridge.static_call( "weka/core/Instances", "mergeInstances", "(Lweka/core/Instances;Lweka/core/Instances;)Lweka/core/Instances;", inst1.jobject, inst2.jobject)) @classmethod def append_instances(cls, inst1, inst2): """ Merges the two datasets (one-after-the-other). Throws an exception if the datasets aren't compatible. :param inst1: the first dataset :type inst1: Instances :param inst2: the first dataset :type inst2: Instances :return: the combined dataset :rtype: Instances """ msg = inst1.equal_headers(inst2) if msg is not None: raise Exception("Cannot appent instances: " + msg) result = cls.copy_instances(inst1) for i in range(inst2.num_instances): result.add_instance(inst2.get_instance(i)) return result def train_test_split(self, percentage, rnd=None): """ Generates a train/test split. :param percentage: the percentage split to use (amount to use for training; 0-100) :type percentage: double :param rnd: the random number generator to use, if None the order gets preserved :type rnd: Random :return: the train/test splits :rtype: tuple """ if percentage <= 0: raise Exception("Split percentage must be > 0, provided: " + str(percentage)) if percentage >= 100: raise Exception("Split percentage must be < 100, provided: " + str(percentage)) if rnd is not None: self.randomize(rnd) train_size = int(round(self.num_instances * percentage / 100)) test_size = self.num_instances - train_size train_inst = Instances.copy_instances(self, 0, train_size) test_inst = Instances.copy_instances(self, train_size, test_size) return train_inst, test_inst @classmethod def summary(cls, inst): """ Generates a summary of the dataset. :param inst: the dataset :type inst: Instances :return: the summary :rtype: str """ return javabridge.call(inst.jobject, "toSummaryString", "()Ljava/lang/String;") class Instance(JavaObject): """ Wrapper class for weka.core.Instance. """ def __init__(self, jobject): """ Initializes the weka.core.Instance wrapper. :param jobject: the weka.core.Instance object to initialize with :type jobject: JB_Object """ self.enforce_type(jobject, "weka.core.Instance") super(Instance, self).__init__(jobject) self.__set_value = javabridge.make_call(self.jobject, "setValue", "(ID)V") self.__get_value = javabridge.make_call(self.jobject, "value", "(I)D") self.__set_string_value = javabridge.make_call(self.jobject, "setValue", "(ILjava/lang/String;)V") self.__get_string_value = javabridge.make_call(self.jobject, "stringValue", "(I)Ljava/lang/String;") self.__set_weight = javabridge.make_call(self.jobject, "setWeight", "(D)V") self.__get_weight = javabridge.make_call(self.jobject, "weight", "()D") self.__is_missing = javabridge.make_call(self.jobject, "isMissing", "(I)Z") self.__class_index = javabridge.make_call(self.jobject, "classIndex", "()I") def __iter__(self): """ Returns an iterator over the values. :return: a value iterator :rtype: InstanceValueIterator """ return InstanceValueIterator(self) @property def dataset(self): """ Returns the dataset that this instance belongs to. :return: the dataset or None if no dataset set :rtype: Instances """ dataset = javabridge.call(self.jobject, "dataset", "()Lweka/core/Instances;") if dataset is None: return None else: return Instances(dataset) @dataset.setter def dataset(self, dataset): """ Sets the dataset that this instance belongs to (for attribute information). :param dataset: the dataset this instance belongs to. :type dataset: Instances """ javabridge.call(self.jobject, "setDataset", "(Lweka/core/Instances;)V", dataset.jobject) @property def num_attributes(self): """ Returns the number of attributes. :return: the numer of attributes :rtype: int """ return javabridge.call(self.jobject, "numAttributes", "()I") @property def num_classes(self): """ Returns the number of class labels. :return: the numer of class labels :rtype: int """ return javabridge.call(self.jobject, "numClasses", "()I") @property def class_attribute(self): """ Returns the currently set class attribute. :return: the class attribute :rtype: Attribute """ return Attribute(javabridge.call(self.jobject, "classAttribute", "()Lweka/core/Attribute;")) @property def class_index(self): """ Returns the currently set class index. :return: the class index, -1 if not set :rtype: int """ return self.__class_index() def has_class(self): """ Returns whether a class attribute is set (convenience method). :return: whether a class attribute is currently set :rtype: bool """ return self.class_index != -1 def set_value(self, index, value): """ Sets the internal value at the specified position (0-based). :param index: the 0-based index of the attribute :type index: int :param value: the internal float value to set :type value: float """ self.__set_value(index, value) def get_value(self, index): """ Returns the internal value at the specified position (0-based). :param index: the 0-based index of the inernal value :type index: int :return: the internal value :rtype: float """ return self.__get_value(index) def set_string_value(self, index, s): """ Sets the string value at the specified position (0-based). :param index: the 0-based index of the inernal value :type index: int :param s: the string value :type s: str """ return self.__set_string_value(index, javabridge.get_env().new_string(s)) def get_string_value(self, index): """ Returns the string value at the specified position (0-based). :param index: the 0-based index of the inernal value :type index: int :return: the string value :rtype: str """ return javabridge.get_env().get_string(self.__get_string_value(index)) def get_relational_value(self, index): """ Returns the relational value at the specified position (0-based). :param index: the 0-based index of the inernal value :type index: int :return: the relational value :rtype: Instances """ return Instances(javabridge.call(self.jobject, "relationalValue", "(I)Lweka/core/Instances;", index)) def set_missing(self, index): """ Sets the attribute at the specified index to missing. :param index: the 0-based index of the attribute :type index: int """ javabridge.call(self.jobject, "setMissing", "(I)V", index) def is_missing(self, index): """ Returns whether the attribute at the specified index is missing. :param index: the 0-based index of the attribute :type index: int :return: whether the value is missing :rtype: bool """ return self.__is_missing(index) def has_missing(self): """ Returns whether at least one attribute has a missing value. :return: whether at least one value is missing :rtype: bool """ return javabridge.call(self.jobject, "hasMissingValue", "()Z") @property def weight(self): """ Returns the currently set weight. :return: the weight :rtype: float """ return self.__get_weight() @weight.setter def weight(self, weight): """ Sets the weight. :param weight: the weight to set :type weight: float """ self.__set_weight(weight) @property def values(self): """ Returns the internal values of this instance. :return: the values as numpy array :rtype: ndarray """ return javabridge.get_env().get_double_array_elements(javabridge.call(self.jobject, "toDoubleArray", "()[D")) @classmethod def create_instance(cls, values, classname="weka.core.DenseInstance", weight=1.0): """ Creates a new instance. :param values: the float values (internal format) to use, numpy array or list. :type values: ndarray or list :param classname: the classname of the instance (eg weka.core.DenseInstance). :type classname: str :param weight: the weight of the instance :type weight: float """ jni_classname = classname.replace(".", "/") if type(values) is list: for i in range(len(values)): values[i] = float(values[i]) values = numpy.array(values) return Instance( javabridge.make_instance( jni_classname, "(D[D)V", weight, javabridge.get_env().make_double_array(values))) @classmethod def create_sparse_instance(cls, values, max_values, classname="weka.core.SparseInstance", weight=1.0): """ Creates a new sparse instance. :param values: the list of tuples (0-based index and internal format float). The indices of the tuples must be in ascending order and "max_values" must be set to the maximum number of attributes in the dataset. :type values: list :param max_values: the maximum number of attributes :type max_values: int :param classname: the classname of the instance (eg weka.core.SparseInstance). :type classname: str :param weight: the weight of the instance :type weight: float """ jni_classname = classname.replace(".", "/") indices = [] vals = [] for (i, v) in values: indices.append(i) vals.append(float(v)) indices = numpy.array(indices, dtype=numpy.int32) vals = numpy.array(vals) return Instance( javabridge.make_instance( jni_classname, "(D[D[II)V", weight, javabridge.get_env().make_double_array(vals), javabridge.get_env().make_int_array(indices), max_values)) @classmethod def missing_value(cls): """ Returns the numeric value that represents a missing value in Weka (NaN). :return: missing value :rtype: float """ return missing_value() class Attribute(JavaObject): """ Wrapper class for weka.core.Attribute. """ def __init__(self, jobject): """ Initializes the weka.core.Attribute wrapper. :param jobject: the JB_Object :type jobject: JB_Object """ self.enforce_type(jobject, "weka.core.Attribute") super(Attribute, self).__init__(jobject) @property def name(self): """ Returns the name of the attribute. :return: the name :rtype: str """ return javabridge.call(self.jobject, "name", "()Ljava/lang/String;") @property def index(self): """ Returns the index of this attribute. :return: the index :rtype: int """ return javabridge.call(self.jobject, "index", "()I") @property def weight(self): """ Returns the weight of the attribute. :return: the weight :rtype: float """ return javabridge.call(self.jobject, "weight", "()D") @weight.setter def weight(self, weight): """ Sets the weight of the attribute. :param weight: the weight of the attribute :type weight: float """ javabridge.call(self.jobject, "setWeight", "(D)V", weight) def index_of(self, label): """ Returns the index of the label in this attribute. :param label: the string label to get the index for :type label: str :return: the 0-based index :rtype: int """ return javabridge.call(self.jobject, "indexOfValue", "(Ljava/lang/String;)I", label) def value(self, index): """ Returns the label for the index. :param index: the 0-based index of the label to return :type index: int :return: the label :rtype: str """ return javabridge.call(self.jobject, "value", "(I)Ljava/lang/String;", index) @property def num_values(self): """ Returns the number of labels. :return: the number of labels :rtype: int """ return javabridge.call(self.jobject, "numValues", "()I") @property def values(self): """ Returns the labels, strings or relation-values. :return: all the values, None if not NOMINAL, STRING, or RELATION :rtype: list """ enm = javabridge.call(self.jobject, "enumerateValues", "()Ljava/util/Enumeration;") if enm is None: return None else: return typeconv.enumeration_to_list(enm) @property def ordering(self): """ Returns the ordering of the attribute. :return: the ordering (ORDERING_SYMBOLIC, ORDERING_ORDERED, ORDERING_MODULO) :rtype: int """ return javabridge.call(self.jobject, "ordering", "()I") @property def type(self): """ Returns the type of the attribute. See weka.core.Attribute Javadoc. :return: the type :rtype: int """ return javabridge.call(self.jobject, "type", "()I") def type_str(self, short=False): """ Returns the type of the attribute as string. :return: the type :rtype: str """ if short: return javabridge.static_call( "weka/core/Attribute", "typeToStringShort", "(Lweka/core/Attribute;)Ljava/lang/String;", self.jobject) else: return javabridge.static_call( "weka/core/Attribute", "typeToString", "(Lweka/core/Attribute;)Ljava/lang/String;", self.jobject) @property def is_averagable(self): """ Returns whether the attribute is averagable. :return: whether averagable :rtype: bool """ return javabridge.call(self.jobject, "isAveragable", "()Z") @property def is_date(self): """ Returns whether the attribute is a date one. :return: whether date attribute :rtype: bool """ return javabridge.call(self.jobject, "isDate", "()Z") @property def is_nominal(self): """ Returns whether the attribute is a nominal one. :return: whether nominal attribute :rtype: bool """ return javabridge.call(self.jobject, "isNominal", "()Z") @property def is_numeric(self): """ Returns whether the attribute is a numeric one (date or numeric). :return: whether numeric attribute :rtype: bool """ return javabridge.call(self.jobject, "isNumeric", "()Z") @property def is_relation_valued(self): """ Returns whether the attribute is a relation valued one. :return: whether relation valued attribute :rtype: bool """ return javabridge.call(self.jobject, "isRelationValued", "()Z") @property def is_string(self): """ Returns whether the attribute is a string attribute. :return: whether string attribute :rtype: bool """ return javabridge.call(self.jobject, "isString", "()Z") @property def date_format(self): """ Returns the format of this data attribute. See java.text.SimpleDateFormat Javadoc. :return: the format string :rtype: str """ return javabridge.call(self.jobject, "getDateFormat", "()Ljava/lang/String;") @property def lower_numeric_bound(self): """ Returns the lower numeric bound of the numeric attribute. :return: the lower bound :rtype: float """ return javabridge.call(self.jobject, "getLowerNumericBound", "()D") @property def upper_numeric_bound(self): """ Returns the upper numeric bound of the numeric attribute. :return: the upper bound :rtype: float """ return javabridge.call(self.jobject, "getUpperNumericBound", "()D") def is_in_range(self, value): """ Checks whether the value is within the bounds of the numeric attribute. :param value: the numeric value to check :type value: float :return: whether between lower and upper bound :rtype: bool """ return javabridge.call(self.jobject, "isInRange", "(D)Z", value) def add_string_value(self, s): """ Adds the string value, returns the index. :param s: the string to add :type s: str :return: the index :rtype: int """ return javabridge.call(self.jobject, "addStringValue", "(Ljava/lang/String;)I", s) def add_relation(self, instances): """ Adds the relation value, returns the index. :param instances: the Instances object to add :type instances: Instances :return: the index :rtype: int """ return javabridge.call(self.jobject, "addRelation", "(Lweka/core/Instances;)I", instances.jobject) def parse_date(self, s): """ Parses the date string and returns the internal format value. :param s: the date string :type s: str :return: the internal format :rtype: float """ return javabridge.call(self.jobject, "parseDate", "(Ljava/lang/String;)D", s) def equals(self, att): """ Checks whether this attributes is the same as the provided one. :param att: the Attribute to check against :type att: Attribute :return: whether the same :rtype: bool """ return javabridge.call(self.jobject, "equals", "(Lweka/core/Attribute;)Z", att.jobject) def equals_msg(self, att): """ Checks whether this attributes is the same as the provided one. Returns None if the same, otherwise error message. :param att: the Attribute to check against :type att: Attribute :return: None if the same, otherwise error message :rtype: str """ return javabridge.call(self.jobject, "equalsMsg", "(Lweka/core/Attribute;)Ljava/lang/String;", att.jobject) def copy(self, name=None): """ Creates a copy of this attribute. :param name: the new name, uses the old one if None :type name: str :return: the copy of the attribute :rtype: Attribute """ if name is None: return Attribute( javabridge.call(self.jobject, "copy", "()Ljava/lang/Object;")) else: return Attribute( javabridge.call(self.jobject, "copy", "(Ljava/lang/String;)Lweka/core/Attribute;", name)) @classmethod def create_numeric(cls, name): """ Creates a numeric attribute. :param name: the name of the attribute :type name: str """ return Attribute( javabridge.make_instance( "weka/core/Attribute", "(Ljava/lang/String;)V", name)) @classmethod def create_date(cls, name, formt="yyyy-MM-dd'T'HH:mm:ss"): """ Creates a date attribute. :param name: the name of the attribute :type name: str :param formt: the date format, see Javadoc for java.text.SimpleDateFormat :type formt: str """ return Attribute( javabridge.make_instance( "weka/core/Attribute", "(Ljava/lang/String;Ljava/lang/String;)V", name, formt)) @classmethod def create_nominal(cls, name, labels): """ Creates a nominal attribute. :param name: the name of the attribute :type name: str :param labels: the list of string labels to use :type labels: list """ return Attribute( javabridge.make_instance( "weka/core/Attribute", "(Ljava/lang/String;Ljava/util/List;)V", name, javabridge.make_list(labels))) @classmethod def create_string(cls, name): """ Creates a string attribute. :param name: the name of the attribute :type name: str """ return Attribute( javabridge.make_instance( "weka/core/Attribute", "(Ljava/lang/String;Ljava/util/List;)V", name, None)) @classmethod def create_relational(cls, name, inst): """ Creates a relational attribute. :param name: the name of the attribute :type name: str :param inst: the structure of the relational attribute :type inst: Instances """ return Attribute( javabridge.make_instance( "weka/core/Attribute", "(Ljava/lang/String;Lweka/core/Instances;)V", name, inst.jobject)) class AttributeStats(JavaObject): """ Container for attribute statistics. """ def __init__(self, jobject): """ Initializes the container. :param jobject: The Java object to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.core.AttributeStats") super(AttributeStats, self).__init__(jobject) @property def distinct_count(self): """ The number of distinct values. :return: The number of distinct values :rtype: int """ return javabridge.get_field(self.jobject, "distinctCount", "I") @property def int_count(self): """ The number of int-like values. :return: The number of int-like values :rtype: int """ return javabridge.get_field(self.jobject, "intCount", "I") @property def missing_count(self): """ The number of missing values. :return: The number of missing values :rtype: int """ return javabridge.get_field(self.jobject, "missingCount", "I") @property def nominal_counts(self): """ Counts of each nominal value. :return: Counts of each nominal value :rtype: ndarray """ return javabridge.get_env().get_int_array_elements( javabridge.get_field(self.jobject, "nominalCounts", "[I")) @property def nominal_weights(self): """ Weight mass for each nominal value. :return: Weight mass for each nominal value :rtype: ndarray """ return javabridge.get_env().get_double_array_elements( javabridge.get_field(self.jobject, "nominalWeights", "[D")) @property def numeric_stats(self): """ Stats on numeric value distributions. :return: Stats on numeric value distributions :rtype: NumericStats """ return Stats(javabridge.get_field(self.jobject, "numericStats", "Lweka/experiment/Stats;")) @property def total_count(self): """ The total number of values. :return: The total number of values :rtype: int """ return javabridge.get_field(self.jobject, "totalCount", "I") @property def unique_count(self): """ The number of values that only appear once. :return: The number of values that only appear once :rtype: int """ return javabridge.get_field(self.jobject, "uniqueCount", "I") class Stats(JavaObject): """ Container for numeric attribute stats. """ def __init__(self, jobject): """ Initializes the container. :param jobject: The Java object to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.experiment.Stats") super(Stats, self).__init__(jobject) @property def count(self): """ The number of values seen. :return: The number of values seen :rtype: float """ return javabridge.get_field(self.jobject, "count", "D") @property def min(self): """ The minimum value seen, or Double.NaN if no values seen. :return: The minimum value seen, or Double.NaN if no values seen :rtype: float """ return javabridge.get_field(self.jobject, "min", "D") @property def max(self): """ The maximum value seen, or Double.NaN if no values seen. :return: The maximum value seen, or Double.NaN if no values seen :rtype: float """ return javabridge.get_field(self.jobject, "max", "D") @property def mean(self): """ The mean of values at the last calculateDerived() call. :return: The mean of values at the last calculateDerived() call :rtype: float """ return javabridge.get_field(self.jobject, "mean", "D") @property def stddev(self): """ The std deviation of values at the last calculateDerived() call. :return: The std deviation of values at the last calculateDerived() call :rtype: float """ return javabridge.get_field(self.jobject, "stdDev", "D") @property def sum(self): """ The sum of values seen. :return: The sum of values seen :rtype: float """ return javabridge.get_field(self.jobject, "sum", "D") @property def sumsq(self): """ The sum of values squared seen. :return: The sum of values squared seen :rtype: float """ return javabridge.get_field(self.jobject, "sumSq", "D") class InstanceIterator(object): """ Iterator for rows in an Instances object. """ def __init__(self, data): """ Initializes the iterator. :param data: the Instances object to iterate over :type data: Instances """ self.data = data self.row = 0 def __iter__(self): """ Returns itself. """ return self def __next__(self): """ Returns the next row from the Instances object. :return: the next Instance object :rtype: Instance """ if self.row < self.data.num_instances: index = self.row self.row += 1 return self.data.get_instance(index) else: raise StopIteration() class AttributeIterator(object): """ Iterator for attributes in an Instances object. """ def __init__(self, data): """ Initializes the iterator. :param data: the Instances object to iterate over :type data: Instances """ self.data = data self.col = 0 def __iter__(self): """ Returns itself. """ return self def __next__(self): """ Returns the next attribute from the Instances object. :return: the next Attribute object :rtype: Attribute """ if self.col < self.data.num_attributes: index = self.col self.col += 1 return self.data.attribute(index) else: raise StopIteration() class InstanceValueIterator(object): """ Iterator for values in an Instance object. """ def __init__(self, data): """ Initializes the iterator. :param data: the Instance object to iterate over :type data: Instance """ self.data = data self.col = 0 def __iter__(self): """ Returns itself. """ return self def __next__(self): """ Returns the next value from the Instance object. :return: the next value, depending on the attribute that can be either a number of a string :rtype: str or float """ if self.col < self.data.num_attributes: index = self.col self.col += 1 if self.data.dataset.attribute(index).is_numeric: return self.data.get_value(index) else: return self.data.get_string_value(index) else: raise StopIteration() def create_instances_from_lists(x, y=None, name="data"): """ Allows the generation of an Instances object from a list of lists for X and a list for Y (optional). All data must be numerical. Attributes can be converted to nominal with the weka.filters.unsupervised.attribute.NumericToNominal filter. :param x: the input variables (row wise) :type x: list of list :param y: the output variable (optional) :type y: list :param name: the name of the dataset :type name: str :return: the generated dataset :rtype: Instances """ if y is not None: if len(x) != len(y): raise Exception("Dimensions of x and y differ: " + str(len(x)) + " != " + str(len(y))) # create header atts = [] for i in range(len(x[0])): atts.append(Attribute.create_numeric("x" + str(i+1))) if y is not None: atts.append(Attribute.create_numeric("y")) result = Instances.create_instances(name, atts, len(x)) # add data for i in range(len(x)): values = x[i][:] if y is not None: values.append(y[i]) result.add_instance(Instance.create_instance(values)) return result def create_instances_from_matrices(x, y=None, name="data"): """ Allows the generation of an Instances object from a 2-dimensional matrix for X and a 1-dimensional matrix for Y (optional). All data must be numerical. Attributes can be converted to nominal with the weka.filters.unsupervised.attribute.NumericToNominal filter. :param x: the input variables :type x: ndarray :param y: the output variable (optional) :type y: ndarray :param name: the name of the dataset :type name: str :return: the generated dataset :rtype: Instances """ if y is not None: if len(x) != len(y): raise Exception("Dimensions of x and y differ: " + str(len(x)) + " != " + str(len(y))) # create header atts = [] for i in range(len(x[0])): atts.append(Attribute.create_numeric("x" + str(i+1))) if y is not None: atts.append(Attribute.create_numeric("y")) result = Instances.create_instances(name, atts, len(x)) # add data for i in range(len(x)): values = list(x[i]) if y is not None: values.append(y[i]) result.add_instance(Instance.create_instance(values)) return result def missing_value(): """ Returns the value that represents missing values in Weka (NaN). :return: missing value :rtype: float """ return float("NaN") python-weka-wrapper3-0.1.3/python/weka/core/classes.py0000644017267001726700000015774013110713652024376 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # classes.py # Copyright (C) 2014-2017 Fracpete (pythonwekawrapper at gmail dot com) import os import inspect import argparse import json import logging import re import weka.core.typeconv as typeconv import javabridge from javabridge import JWrapper, JClassWrapper from javabridge.jutil import JavaException import weka.core.jvm as jvm # logging setup logger = logging.getLogger("weka.core.classes") def get_class(classname): """ Returns the class object associated with the dot-notation classname. Taken from here: http://stackoverflow.com/a/452981 :param classname: the classname :type classname: str :return: the class object :rtype: object """ parts = classname.split('.') module = ".".join(parts[:-1]) m = __import__(module) for comp in parts[1:]: m = getattr(m, comp) return m def get_jclass(classname): """ Returns the Java class object associated with the dot-notation classname. :param classname: the classname :type classname: str :return: the class object :rtype: JB_Object """ return javabridge.class_for_name(classname=classname) def get_classname(obj): """ Returns the classname of the JB_Object, Python class or object. :param obj: the java object or Python class/object to get the classname for :type obj: object :return: the classname :rtype: str """ if isinstance(obj, javabridge.JB_Object): cls = javabridge.call(obj, "getClass", "()Ljava/lang/Class;") return javabridge.call(cls, "getName", "()Ljava/lang/String;") elif inspect.isclass(obj): return obj.__module__ + "." + obj.__name__ else: return get_classname(obj.__class__) class Stoppable(object): """ Classes that can be stopped. """ def is_stopped(self): """ Returns whether the object has been stopped. :return: whether stopped :rtype: bool """ raise Exception("Not implemented!") def stop_execution(self): """ Triggers the stopping of the object. """ raise Exception("Not implemented!") from_dict_handlers = {} """ The methods that handle the restoration from a JSON dictionary, stored under their 'type'. """ def register_dict_handler(typestr, handler): """ Registers a handler for restoring an object from a JSON dictionary. :param typestr: the type of the object :type typestr: str :param handler: the method """ global from_dict_handlers from_dict_handlers[typestr] = handler def deregister_dict_handler(typestr): """ Deregisters a handler for restoring an object from a JSON dictionary. :param typestr: the type of the object :type typestr: str """ global from_dict_handlers del from_dict_handlers[typestr] def has_dict_handler(typestr): """ Returns the handler for restoring an object from a JSON dictionary. :param typestr: the type of the object :type typestr: str :return: the handler, None if not available """ global from_dict_handlers return typestr in from_dict_handlers def get_dict_handler(typestr): """ Returns the handler for restoring an object from a JSON dictionary. :param typestr: the type of the object :type typestr: str :return: the handler, None if not available """ global from_dict_handlers return from_dict_handlers[str(typestr)] class JSONObject(object): """ Ancestor for classes that can be represented as JSON and restored from JSON. """ def to_dict(self): """ Returns a dictionary that represents this object, to be used for JSONification. :return: the object dictionary :rtype: dict """ raise Exception("Not implemented!") @classmethod def from_dict(cls, d): """ Restores an object state from a dictionary, used in de-JSONification. :param d: the object dictionary :type d: dict :return: the object :rtype: object """ raise Exception("Not implemented!") def to_json(self): """ Returns the options as JSON. :return: the object as string :rtype: str """ return json.dumps(self.to_dict(), sort_keys=True, indent=2, separators=(',', ': ')) @classmethod def from_json(cls, s): """ Restores the object from the given JSON. :param s: the JSON string to parse :type s: str :return: the """ d = json.loads(s) return get_dict_handler(d["type"])(d) def shallow_copy(self): """ Returns a shallow copy of itself. :return: the copy :rtype: object """ return self.from_json(self.to_json()) class Configurable(JSONObject): """ The ancestor for all actors. """ def __init__(self, config=None): """ Initializes the object. :param config: the dictionary with the options (str -> object). :type config: dict """ self._logger = None self._help = {} self._config = self.fix_config({}) if config is not None: self.config = config if not has_dict_handler("Configurable"): register_dict_handler("Configurable", Configurable.from_dict) def __repr__(self): """ Returns Python code for instantiating the object. :return: the representation :rtype: str """ return \ self.__class__.__module__ + "." + self.__class__.__name__ \ + "(config=" + str(self.config) + ")" def description(self): """ Returns a description of the object. :return: the description :rtype: str """ raise Exception("Not implemented!") def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ return options @property def config(self): """ Obtains the currently set options of the actor. :return: the options :rtype: dict """ return self._config @config.setter def config(self, options): """ Sets the options of the actor. :param options: the options :type options: dict """ self._config = self.fix_config(options) def to_dict(self): """ Returns a dictionary that represents this object, to be used for JSONification. :return: the object dictionary :rtype: dict """ result = {} result["type"] = "Configurable" result["class"] = get_classname(self) result["config"] = {} for k in self._config: v = self._config[k] if isinstance(v, JSONObject): result["config"][k] = v.to_dict() else: result["config"][k] = v return result @classmethod def from_dict(cls, d): """ Restores its state from a dictionary, used in de-JSONification. :param d: the object dictionary :type d: dict """ conf = {} for k in d["config"]: v = d["config"][k] if isinstance(v, dict): conf[str(k)] = get_dict_handler(d["config"]["type"])(v) else: conf[str(k)] = v return get_class(str(d["class"]))(config=conf) def new_logger(self): """ Returns a new logger instance. :return: the logger instance :rtype: logger """ return logging.getLogger(get_classname(self)) @property def logger(self): """ Returns the logger object. :return: the logger :rtype: logger """ if self._logger is None: self._logger = self.new_logger() return self._logger @property def help(self): """ Obtains the help information per option for this actor. :return: the help :rtype: dict """ return self._help def generate_help(self): """ Generates a help string for this actor. :return: the help string :rtype: str """ result = [] result.append(self.__class__.__name__) result.append(re.sub(r'.', '=', self.__class__.__name__)) result.append("") result.append("DESCRIPTION") result.append(self.description()) result.append("") result.append("OPTIONS") opts = sorted(self.config.keys()) for opt in opts: result.append(opt) helpstr = self.help[opt] if helpstr is None: helpstr = "-missing help-" result.append("\t" + helpstr) result.append("") return '\n'.join(result) def print_help(self): """ Prints a help string for this actor to stdout. """ print(self.generate_help()) class JavaObject(JSONObject): """ Basic Java object. """ def __init__(self, jobject): """ Initializes the wrapper with the specified Java object. :param jobject: the Java object to wrap :type jobject: JB_Object """ if jobject is None: raise Exception("No Java object supplied!") self.jobject = jobject if not has_dict_handler("JavaObject"): register_dict_handler("JavaObject", JavaObject.from_dict) def __str__(self): """ Just calls the toString() method. :rtype: str """ return javabridge.to_string(self.jobject) def __repr__(self): """ Just calls the toString() method. :rtype: str """ return javabridge.to_string(self.jobject) def __unicode__(self): """ Just calls the toString() method. :rtype: str """ return javabridge.to_string(self.jobject) @property def classname(self): """ Returns the Java classname in dot-notation. :return: the Java classname :rtype: str """ cls = javabridge.call(self.jobject, "getClass", "()Ljava/lang/Class;") return javabridge.call(cls, "getName", "()Ljava/lang/String;") def set_property(self, path, jobject): """ Attempts to set the value (jobject, a Java object) of the provided (bean) property path. :param path: the property path, e.g., "filter" for a setFilter(...)/getFilter() method pair :type path: str :param jobject: the Java object to set; if instance of JavaObject class, the jobject member is automatically used :type jobject: JB_Object """ # unwrap? if isinstance(jobject, JavaObject): jobject = jobject.jobject javabridge.static_call( "Lweka/core/PropertyPath;", "setValue", "(Ljava/lang/Object;Ljava/lang/String;Ljava/lang/Object;)V", self.jobject, path, jobject) def get_property(self, path): """ Attempts to get the value (jobject, a Java object) of the provided (bean) property path. :param path: the property path, e.g., "filter" for a setFilter(...)/getFilter() method pair :type path: str :return: the wrapped Java object :rtype: JavaObject """ return JavaObject(javabridge.static_call( "Lweka/core/PropertyPath;", "getValue", "(Ljava/lang/Object;Ljava/lang/String;)Ljava/lang/Object;", self.jobject, path)) @property def jclass(self): """ Returns the Java class object of the underlying Java object. :return: the Java class :rtype: JB_Object """ return javabridge.call(self.jobject, "getClass", "()Ljava/lang/Class;") @property def jwrapper(self): """ Returns a JWrapper instance of the encapsulated Java object, giving access to methods using dot notation. http://pythonhosted.org//javabridge/highlevel.html#wrapping-java-objects-using-reflection :return: the wrapper :rtype: JWrapper """ return JWrapper(self.jobject) @property def jclasswrapper(self): """ Returns a JClassWrapper instance of the class for the encapsulated Java object, giving access to the class methods using dot notation. http://pythonhosted.org//javabridge/highlevel.html#wrapping-java-objects-using-reflection :return: the wrapper :rtype: JClassWrapper """ return JClassWrapper(javabridge.call(self.jobject, "getClass", "()Ljava/lang/Class;")) @property def is_serializable(self): """ Returns true if the object is serialiable. :return: true if serializable :rtype: bool """ return JavaObject.check_type(self.jobject, "java.io.Serializable") def to_dict(self): """ Returns a dictionary that represents this object, to be used for JSONification. :return: the object dictionary :rtype: dict """ result = {} result["type"] = "JavaObject" result["class"] = self.classname return result @classmethod def from_dict(cls, d): """ Restores an object state from a dictionary, used in de-JSONification. :param d: the object dictionary :type d: dict :return: the object :rtype: object """ return JavaObject(cls.new_instance(d["class"])) @classmethod def check_type(cls, jobject, intf_or_class, jni_intf_or_class=None): """ Returns whether the object implements the specified interface or is a subclass. E.g.: self._check_type('weka.core.OptionHandler', 'Lweka/core/OptionHandler;') or self._check_type('weka.core.converters.AbstractFileLoader') :param jobject: the Java object to check :type jobject: JB_Object :param intf_or_class: the classname in Java notation (eg "weka.core.DenseInstance;") :type jni_intf_or_class: str :return: whether object implements interface or is subclass :rtype: bool """ if jni_intf_or_class is None: if intf_or_class.startswith("."): intf_or_class = complete_classname(intf_or_class) jni_intf_or_class = "L" + intf_or_class.replace(".", "/") + ";" return javabridge.is_instance_of(jobject, jni_intf_or_class) @classmethod def enforce_type(cls, jobject, intf_or_class, jni_intf_or_class=None): """ Raises an exception if the object does not implement the specified interface or is not a subclass. E.g.: self._enforce_type('weka.core.OptionHandler', 'Lweka/core/OptionHandler;') or self._enforce_type('weka.core.converters.AbstractFileLoader') :param jobject: the Java object to check :type jobject: JB_Object :param intf_or_class: the classname in Java notation (eg "weka.core.DenseInstance") :type intf_or_class: str :param jni_intf_or_class: the classname in JNI notation (eg "Lweka/core/DenseInstance;") :type jni_intf_or_class: str """ if not cls.check_type(jobject, intf_or_class, jni_intf_or_class): raise TypeError("Object does not implement or subclass " + intf_or_class + ": " + get_classname(jobject)) @classmethod def new_instance(cls, classname, jni_classname=None): """ Creates a new object from the given classname using the default constructor, None in case of error. :param classname: the classname in Java notation (eg "weka.core.DenseInstance") :type classname: str :param jni_classname: the classname in JNI notation (eg "Lweka/core/DenseInstance;") :type jni_classname: str :return: the Java object :rtype: JB_Object """ if jni_classname is None: if classname.startswith("."): classname = complete_classname(classname) jni_classname = classname.replace(".", "/") try: return javabridge.make_instance(jni_classname, "()V") except JavaException as e: print("Failed to instantiate " + classname + "/" + jni_classname + ": " + str(e)) return None class JavaArrayIterator(object): """ Iterator for elements in a Java array. """ def __init__(self, data): """ :param data: the Java array to iterate over :type data: JavaArray """ self.data = data self.index = 0 self.length = len(data) def __iter__(self): """ Returns itself. """ return self def __next__(self): """ Returns the next element from the array. :return: the next array element object, wrapped as JavaObject if not null :rtype: JavaObject or None """ if self.index < self.length: index = self.index self.index += 1 return self.data[index] else: raise StopIteration() class JavaArray(JavaObject): """ Convenience wrapper around Java arrays. """ def __init__(self, jobject): """ Initializes the wrapper with the specified Java object. :param jobject: the java array object to wrap :type jobject: JB_Object """ super(JavaArray, self).__init__(jobject) c = self.jclass if not javabridge.call(c, "isArray", "()Z"): raise Exception("Not an array!") def __len__(self): """ Returns the length of the array. :return: the array length :rtype: int """ return javabridge.get_env().get_array_length(self.jobject) def __getitem__(self, key): """ Returns the specified element in the array wrapped in a JavaObject. :param key: the index of the element to retrieve :type key: int :return: the element or None if element is null :rtype; JavaObject """ if not isinstance(key, int): raise Exception("Key must be an integer!") element = javabridge.static_call( "Ljava/lang/reflect/Array;", "get", "(Ljava/lang/Object;I)Ljava/lang/Object;", self.jobject, key) if element is None: return None return JavaObject(element) def __setitem__(self, key, value): """ Sets the specified element in the array. :param key: the index of the element to set :type key: int :param value: the object to set (JavaObject or JB_Object) """ if isinstance(value, JavaObject): obj = value.jobject else: obj = value if not isinstance(key, int): raise Exception("Key must be an integer!") javabridge.static_call( "Ljava/lang/reflect/Array;", "set", "(Ljava/lang/Object;ILjava/lang/Object;)V", self.jobject, key, obj) def __delitem__(self, key): """ Not implemented, raises an exception. """ raise Exception("Cannot delete item from array!") def __iter__(self): """ Returns an iterator over the elements. :return: the iterator :rtype: JavaArrayIterator """ return JavaArrayIterator(self) @classmethod def new_instance(cls, classname, length): """ Creates a new array with the given classname and length; initial values are null. :param classname: the classname in Java notation (eg "weka.core.DenseInstance") :type classname: str :param length: the length of the array :type length: int :return: the Java array :rtype: JB_Object """ return javabridge.static_call( "Ljava/lang/reflect/Array;", "newInstance", "(Ljava/lang/Class;I)Ljava/lang/Object;", javabridge.class_for_name(classname=classname), length) class Enum(JavaObject): """ Wrapper for Java enums. """ def __init__(self, jobject=None, enum=None, member=None): """ Initializes the enum class. :param jobject: the Java object to wrap :type jobject: JB_Object :param enum: the enum class to instantiate (dot notation) :type enum: str :param member: the member of the enum class to instantiate :type member: str """ if jobject is None: enumclass = javabridge.class_for_name(classname=enum) jobject = javabridge.static_call( "java/lang/Enum", "valueOf", "(Ljava/lang/Class;Ljava/lang/String;)Ljava/lang/Enum;", enumclass, member) super(Enum, self).__init__(jobject) @property def name(self): """ Returns the name of the enum member. :return: the name :rtype: str """ return javabridge.call(self.jobject, "name", "()Ljava/lang/String;") @property def ordinal(self): """ Returns the ordinal of the enum member. :return: the ordinal :rtype: int """ return javabridge.call(self.jobject, "ordinal", "()I") @property def values(self): """ Returns list of all enum members. :return: all enum members :rtype: list """ cls = javabridge.call(self.jobject, "getClass", "()Ljava/lang/Class;") clsname = javabridge.call(cls, "getName", "()Ljava/lang/String;") l = javabridge.static_call(clsname.replace(".", "/"), "values", "()[L" + clsname.replace(".", "/") + ";") l = javabridge.get_env().get_object_array_elements(l) result = [] for item in l: result.append(Enum(jobject=item)) return result class Random(JavaObject): """ Wrapper for the java.util.Random class. """ def __init__(self, seed): """ The seed value. :param seed: the seed value :type seed: int """ super(Random, self).__init__(javabridge.make_instance("Ljava/util/Random;", "(J)V", seed)) def next_int(self, n=None): """ Next random integer. if n is provided, then between 0 and n-1. :param n: the upper limit (minus 1) for the random integer :type n: int :return: the next random integer :rtype: int """ if n is None: return javabridge.call(self.jobject, "nextInt", "()I") else: return javabridge.call(self.jobject, "nextInt", "(I)I", n) def next_double(self): """ Next random double. :return: the next random double :rtype: double """ return javabridge.call(self.jobject, "nextDouble", "()D") class Option(JavaObject): """ Wrapper for the weka.core.Option class. """ def __init__(self, jobject): """ Initializes the wrapper with the specified option object. :param jobject: the java object to wrap :type jobject: JB_Object """ super(Option, self).__init__(jobject) Option.enforce_type(jobject, "weka.core.Option") @property def name(self): """ Returns the name of the option. :return: the name :rtype: str """ return javabridge.call(self.jobject, "name", "()Ljava/lang/String;") @property def description(self): """ Returns the description of the option. :return: the description :rtype: str """ return javabridge.call(self.jobject, "description", "()Ljava/lang/String;") @property def synopsis(self): """ Returns the synopsis of the option. :return: the synopsis :rtype: str """ return javabridge.call(self.jobject, "synopsis", "()Ljava/lang/String;") @property def num_arguments(self): """ Returns the synopsis of the option. :return: the synopsis :rtype: str """ return javabridge.call(self.jobject, "numArguments", "()I") class OptionHandler(JavaObject, Configurable): """ Ancestor for option-handling classes. Classes should implement the weka.core.OptionHandler interface to have any effect. """ def __init__(self, jobject, options=None): """ Initializes the wrapper with the specified Java object. :param jobject: the java object to wrap :type jobject: JB_Object :param options: the options to set :type options: list """ super(OptionHandler, self).__init__(jobject) self.is_optionhandler = OptionHandler.check_type(jobject, "weka.core.OptionHandler") if (options is not None) and (len(options) > 0): self.options = options # we have to manually instantiate some objects, since multiple inheritance doesn't call # Configurable constructor self._logger = None self._help = {} self._config = self.fix_config({}) if not has_dict_handler("OptionHandler"): register_dict_handler("OptionHandler", OptionHandler.from_dict) def global_info(self): """ Returns the globalInfo() result, None if not available. :rtypes: str """ try: return javabridge.call(self.jobject, "globalInfo", "()Ljava/lang/String;") except JavaException: return None def description(self): """ Returns a description of the object. :return: the description :rtype: str """ return self.global_info() @property def options(self): """ Obtains the currently set options as list. :return: the list of options :rtype: list """ if self.is_optionhandler: return typeconv.string_array_to_list(javabridge.call(self.jobject, "getOptions", "()[Ljava/lang/String;")) else: return [] @options.setter def options(self, options): """ Sets the command-line options (as list). :param options: the list of command-line options to set :type options: list """ if self.is_optionhandler: javabridge.call(self.jobject, "setOptions", "([Ljava/lang/String;)V", typeconv.string_list_to_array(options)) def to_commandline(self): """ Generates a commandline string from the JavaObject instance. :return: the commandline string :rtype: str """ return javabridge.static_call( "Lweka/core/Utils;", "toCommandLine", "(Ljava/lang/Object;)Ljava/lang/String;", self.jobject) def to_help(self): """ Returns a string that contains the 'global_info' text and the options. :return: the generated help string :rtype: str """ result = [] result.append(self.classname) result.append("=" * len(self.classname)) result.append("") result.append("DESCRIPTION") result.append("") result.append(self.global_info()) result.append("") result.append("OPTIONS") result.append("") options = javabridge.call(self.jobject, "listOptions", "()Ljava/util/Enumeration;") enum = javabridge.get_enumeration_wrapper(options) while enum.hasMoreElements(): opt = Option(enum.nextElement()) result.append(opt.synopsis) result.append(opt.description) result.append("") return '\n'.join(result) def __str__(self): """ Calls the toString() method of the java object. :return: the result of the toString() method :rtype: str """ return javabridge.to_string(self.jobject) def to_dict(self): """ Returns a dictionary that represents this object, to be used for JSONification. :return: the object dictionary :rtype: dict """ result = super(OptionHandler, self).to_dict() result["type"] = "OptionHandler" result["options"] = join_options(self.options) return result @classmethod def from_dict(cls, d): """ Restores an object state from a dictionary, used in de-JSONification. :param d: the object dictionary :type d: dict :return: the object :rtype: object """ result = OptionHandler(cls.new_instance(d["class"])) result.options = split_options(d["options"]) return result class SingleIndex(JavaObject): """ Wrapper for a Weka SingleIndex object. """ def __init__(self, jobject=None, index=None): """ Initializes the wrapper with the specified Java object or string index. :param jobject: the java object to wrap :type jobject: JB_Object :param index: the string index to use :type index: str """ if jobject is None: if index is None: jobject = javabridge.make_instance("weka/core/SingleIndex", "()V") else: jobject = javabridge.make_instance("weka/core/SingleIndex", "(Ljava/lang/String;)V", index) else: self.enforce_type(jobject, "weka.core.SingleIndex") super(SingleIndex, self).__init__(jobject) def upper(self, upper): """ Sets the upper limit. :param upper: the upper limit :type upper: int """ javabridge.call(self.jobject, "setUpper", "(I)V", upper) def index(self): """ Returns the integer index. :return: the 0-based integer index :rtype: int """ return javabridge.call(self.jobject, "getIndex", "()I") @property def single_index(self): """ Returns the string index. :return: the 1-based string index :rtype: str """ return javabridge.call(self.jobject, "getSingleIndex", "()Ljava/lang/String;") @single_index.setter def single_index(self, index): """ Sets the string index. :param index: the 1-based string index ::type index: str """ javabridge.call(self.jobject, "setSingleIndex", "(Ljava/lang/String;)V", index) class Range(JavaObject): """ Wrapper for a Weka Range object. """ def __init__(self, jobject=None, ranges=None): """ Initializes the wrapper with the specified Java object or string range. :param jobject: the java object to wrap :type jobject: JB_Object :param ranges: the string range to use :type ranges: str """ if jobject is None: if ranges is None: jobject = javabridge.make_instance("weka/core/Range", "()V") else: jobject = javabridge.make_instance("weka/core/Range", "(Ljava/lang/String;)V", ranges) else: self.enforce_type(jobject, "weka.core.Range") super(Range, self).__init__(jobject) def upper(self, upper): """ Sets the upper limit. :param upper: the upper limit :type upper: int """ javabridge.call(self.jobject, "setUpper", "(I)V", upper) def selection(self): """ Returns the selection list. :return: the list of 0-based integer indices :rtype: list """ return javabridge.get_env().get_int_array_elements(javabridge.call(self.jobject, "getSelection", "()[I")) @property def ranges(self): """ Returns the string range. :return: the string range of 1-based indices :rtype: str """ return javabridge.call(self.jobject, "getRanges", "()Ljava/lang/String;") @ranges.setter def ranges(self, rng): """ Sets the string range. :param rng: the range to set :type rng: str """ javabridge.call(self.jobject, "setRanges", "(Ljava/lang/String;)V", rng) @property def invert(self): """ Returns whether the range is inverted. :return: true if inverted :rtype: bool """ return javabridge.call(self.jobject, "getInvert", "()Z") @invert.setter def invert(self, invert): """ Sets the invert state. :param invert: whether to invert or not :type invert: bool """ javabridge.call(self.jobject, "setInvert", "(Z)V", invert) class Tag(JavaObject): """ Wrapper for the weka.core.Tag class. """ def __init__(self, jobject=None, ident=None, ident_str="", readable="", uppercase=True): """ :param jobject: the java object to wrap :type jobject: JB_Object :param ident: the ID integer associated with the tag :type ident: int :param ident_str: the ID string associated with the tag (case-insensitive) :type ident_str: str :param readable: the text of the tag :type readable: str :param uppercase: whether to uppercase the id string :type uppercase: bool """ if jobject is None: jobject = javabridge.make_instance( "weka/core/Tag", "(ILjava/lang/String;Ljava/lang/String;Z)V", ident, ident_str, readable, uppercase) else: self.enforce_type(jobject, "weka.core.Tag") super(Tag, self).__init__(jobject) @property def ident(self): """ Returns the current integer ID of the tag. :return: the integer ID :rtype: int """ return javabridge.call(self.jobject, "getID", "()I") @ident.setter def ident(self, value): """ Sets the integer ID of the tag. :param value: the new ID :type value: int """ javabridge.call(self.jobject, "setID", "(I)V", value) @property def identstr(self): """ Returns the current ID string. :return: the ID string :rtype: str """ return javabridge.call(self.jobject, "getIDStr", "()Ljava/lang/String;") @identstr.setter def identstr(self, value): """ Sets the ID string. :param value: the new ID string :type value: str """ javabridge.call(self.jobject, "setIDStr", "(Ljava/lang/String;)V", value) @property def readable(self): """ Returns the 'human readable' string. :return: the readable string :rtype: str """ return javabridge.call(self.jobject, "getReadable", "()Ljava/lang/String;") @readable.setter def readable(self, value): """ Sets the 'human readable' string. :param value: the new readable string :type value: str """ javabridge.call(self.jobject, "setReadable", "(Ljava/lang/String;)V", value) class Tags(JavaObject): """ Wrapper for an array of weka.core.Tag objects. """ def __init__(self, jobject=None, tags=None): """ :param jobject: the Java Tag array to wrap. :type jobject: JB_Object :param tags: the list of Tag objects to use :type tags: list """ if tags is not None: jarray = JavaArray(JavaArray.new_instance("weka.core.Tag", len(tags))) for i in range(len(tags)): jarray[i] = tags[i] jobject = jarray.jobject self.enforce_type(jobject, "weka.core.Tag", jni_intf_or_class="[Lweka/core/Tag;") super(Tags, self).__init__(jobject) self.array = JavaArray(self.jobject) def __len__(self): """ Returns the number of Tag objects in the array. :return: the number of tag objects :rtype: int """ return len(self.array) def __getitem__(self, item): """ Returns the specified Tag from the array. :param item: the 0-based index :type item: int :return: the tag :rtype: Tag """ return Tag(self.array[item].jobject) def __setitem__(self, key, value): """ Not implemented. """ raise Exception("Cannot set a Tag!") def __delitem__(self, key): """ Not implemented. """ raise Exception("Cannot delete a Tag!") def __str__(self): """ Just calls the toString() method. :rtype: str """ result = "" for i in range(len(self.array)): if i > 0: result += "|" result += str(self.array[i]) return result def find(self, name): """ Returns the Tag that matches the name. :param name: the string representation of the tag :type name: str :return: the tag, None if not found :rtype: Tag """ result = None for t in self.array: if str(t) == name: result = Tag(t.jobject) break return result @classmethod def get_tags(cls, classname, field): """ Instantiates the Tag array located in the specified class with the given field name. Example: tags = Tags.get_tags("weka.classifiers.functions.SMO", "TAGS_FILTER") :param classname: the classname in which the tags reside :type classname: str :param field: the field name of the Tag array :type field: str :return: the Tags objects :rtype: Tags """ classname = "L" + classname.replace(".", "/") + ";" return Tags(jobject=javabridge.get_static_field(classname, field, "[Lweka/core/Tag;")) @classmethod def get_object_tags(cls, javaobject, methodname): """ Instantiates the Tag array obtained from the object using the specified method name. Example: cls = Classifier(classname="weka.classifiers.meta.MultiSearch") tags = Tags.get_object_tags(cls, "getMetricsTags") :param javaobject: the javaobject to obtain the tags from :type javaobject: JavaObject :param methodname: the method name returning the Tag array :type methodname: str :return: the Tags objects :rtype: Tags """ return Tags(jobject=javabridge.call(javaobject.jobject, methodname, "()[Lweka/core/Tag;")) class SelectedTag(JavaObject): """ Wrapper for the weka.core.SelectedTag class. """ def __init__(self, jobject=None, tag_id=None, tag_text=None, tags=None): """ Initializes the wrapper with the specified Java object or tags and either tag_id or tag_text. :param jobject: the java object to wrap :type jobject: JB_Object :param tag_id: the integer associated with the tag :type tag_id: int :param tag_text: the text associated with the tag :type tag_text: str :param tags: list of Tag objects or Tags wrapper object :type tags: list or Tags """ if isinstance(tags, Tags): tobj = tags.jobject else: tobj = tags if jobject is None: if tag_id is None: jobject = javabridge.make_instance( "weka/core/SelectedTag", "(Ljava/lang/String;[Lweka/core/Tag;)V", tag_text, tobj) else: jobject = javabridge.make_instance( "weka/core/SelectedTag", "(I[Lweka/core/Tag;)V", tag_id, tobj) else: self.enforce_type(jobject, "weka.core.SelectedTag") super(SelectedTag, self).__init__(jobject) @property def selected(self): """ Returns the selected tag. :return: the tag :rtype: Tag """ return Tag(javabridge.call(self.jobject, "getSelectedTag", "()Lweka/core/Tag;")) @property def tags(self): """ Returns the associated tags. :return: the list of Tag objects :rtype: list """ result = [] a = javabridge.call(self.jobject, "getTags", "()Lweka/core/Tag;]") length = javabridge.get_env().get_array_length(a) wrapped = javabridge.get_env().get_object_array_elements(a) for i in range(length): result.append(Tag(javabridge.get_env().get_string(wrapped[i]))) return result def join_options(options): """ Turns the list of options back into a single commandline string. :param options: the list of options to process :type options: list :return: the combined options :rtype: str """ return javabridge.static_call( "Lweka/core/Utils;", "joinOptions", "([Ljava/lang/String;)Ljava/lang/String;", options) def split_options(cmdline): """ Splits the commandline into a list of options. :param cmdline: the commandline string to split into individual options :type cmdline: str :return: the split list of commandline options :rtype: list """ return typeconv.string_array_to_list( javabridge.static_call( "Lweka/core/Utils;", "splitOptions", "(Ljava/lang/String;)[Ljava/lang/String;", cmdline)) def backquote(s): """ Backquotes the string. :param s: the string to process :type s: str :return: the backquoted string :rtype: str """ return javabridge.static_call( "Lweka/core/Utils;", "backQuoteChars", "(Ljava/lang/String;)Ljava/lang/String;", s) def unbackquote(s): """ Un-backquotes the string. :param s: the string to process :type s: str :return: the un-backquoted string :rtype: str """ return javabridge.static_call( "Lweka/core/Utils;", "unbackQuoteChars", "(Ljava/lang/String;)Ljava/lang/String;", s) def quote(s): """ Quotes the string if necessary. :param s: the string to process :type s: str :return: the quoted string :rtype: str """ return javabridge.static_call( "Lweka/core/Utils;", "quote", "(Ljava/lang/String;)Ljava/lang/String;", s) def unquote(s): """ Un-quotes the string. :param s: the string to process :type s: str :return: the un-quoted string :rtype: str """ return javabridge.static_call( "Lweka/core/Utils;", "unquote", "(Ljava/lang/String;)Ljava/lang/String;", s) def to_commandline(optionhandler): """ Generates a commandline string from the OptionHandler instance. :param optionhandler: the OptionHandler instance to turn into a commandline :type optionhandler: OptionHandler :return: the commandline string :rtype: str """ return javabridge.static_call( "Lweka/core/Utils;", "toCommandLine", "(Ljava/lang/Object;)Ljava/lang/String;", optionhandler.jobject) def from_commandline(cmdline, classname=None): """ Creates an OptionHandler based on the provided commandline string. :param cmdline: the commandline string to use :type cmdline: str :param classname: the classname of the wrapper to return other than OptionHandler (in dot-notation) :type classname: str :return: the generated option handler instance :rtype: object """ params = split_options(cmdline) cls = params[0] params = params[1:] handler = OptionHandler(jobject=javabridge.make_instance(cls.replace(".", "/"), "()V"), options=params) if classname is None: return handler else: c = get_class(classname) return c(jobject=handler.jobject) def complete_classname(classname): """ Attempts to complete a partial classname like '.J48' and returns the full classname if a single match was found, otherwise an exception is raised. :param classname: the partial classname to expand :type classname: str :return: the full classname :rtype: str """ result = javabridge.get_collection_wrapper( javabridge.static_call( "Lweka/Run;", "findSchemeMatch", "(Ljava/lang/String;Z)Ljava/util/List;", classname, True)) if len(result) == 1: return str(result[0]) elif len(result) == 0: raise Exception("No classname matches found for: " + classname) else: matches = [] for i in range(len(result)): matches.append(str(result[i])) raise Exception("Found multiple matches for '" + classname + "':\n" + '\n'.join(matches)) class AbstractParameter(OptionHandler): """ Ancestor for all parameter classes used by SetupGenerator and MultiSearch. """ def __init__(self, classname=None, jobject=None, options=None): """ Initializes the specified parameter using either the classname or the supplied JB_Object. :param classname: the classname of the search parameter :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = AbstractParameter.new_instance(classname) self.enforce_type(jobject, "weka.core.setupgenerator.AbstractParameter") super(AbstractParameter, self).__init__(jobject=jobject, options=options) @property def prop(self): """ Returns the currently set property to apply the parameter to. :return: the property :rtype: str """ return javabridge.call(self.jobject, "getProperty", "()Ljava/lang/String;") @prop.setter def prop(self, s): """ Sets the property to apply the parameter to. :param s: the property :type s: str """ javabridge.call(self.jobject, "setProperty", "(Ljava/lang/String;)V", s) class ListParameter(AbstractParameter): """ Parameter using a predefined list of values, used by SetupGenerator and MultiSearch. """ def __init__(self, jobject=None, options=None): """ Initializes the specified parameter using either its classname or the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ classname = "weka.core.setupgenerator.ListParameter" if jobject is None: jobject = ListParameter.new_instance(classname) self.enforce_type(jobject, "weka.core.setupgenerator.ListParameter") super(ListParameter, self).__init__(jobject=jobject, options=options) @property def values(self): """ Returns the currently set values. :return: the list of values (strings) :rtype: list """ return split_options(javabridge.call(self.jobject, "getList", "()Ljava/lang/String;")) @values.setter def values(self, l): """ Sets the list of values to apply. :param l: the list of values (strings) :type l: list """ javabridge.call(self.jobject, "setList", "(Ljava/lang/String;)V", join_options(l)) class MathParameter(AbstractParameter): """ Parameter using a math expression for generating values, used by SetupGenerator and MultiSearch. """ def __init__(self, jobject=None, options=None): """ Initializes the specified parameter using either its classname or the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ classname = "weka.core.setupgenerator.MathParameter" if jobject is None: jobject = ListParameter.new_instance(classname) self.enforce_type(jobject, "weka.core.setupgenerator.MathParameter") super(MathParameter, self).__init__(jobject=jobject, options=options) @property def minimum(self): """ Returns the currently set minimum value. :return: the minimum :rtype: float """ return javabridge.call(self.jobject, "getMin", "()D") @minimum.setter def minimum(self, m): """ Sets the new minimum value. :param m: the minimum :type m: float """ javabridge.call(self.jobject, "setMin", "(D)V", m) @property def maximum(self): """ Returns the currently set maximum value. :return: the maximum :rtype: float """ return javabridge.call(self.jobject, "getMax", "()D") @maximum.setter def maximum(self, m): """ Sets the new maximum value. :param m: the maximum :type m: float """ javabridge.call(self.jobject, "setMax", "(D)V", m) @property def step(self): """ Returns the currently set step value. :return: the step :rtype: float """ return javabridge.call(self.jobject, "getStep", "()D") @step.setter def step(self, s): """ Sets the new step value. :param s: the step :type s: float """ javabridge.call(self.jobject, "setStep", "(D)V", s) @property def base(self): """ Returns the currently set base value. :return: the base :rtype: float """ return javabridge.call(self.jobject, "getBase", "()D") @base.setter def base(self, b): """ Sets the new base value. :param b: the base :type b: float """ javabridge.call(self.jobject, "setBase", "(D)V", b) @property def expression(self): """ Returns the currently set expression. :return: the expression :rtype: str """ return javabridge.call(self.jobject, "getExpression", "()Ljava/lang/String;") @expression.setter def expression(self, e): """ Sets the new expression. :param e: the expression :type e: str """ javabridge.call(self.jobject, "setExpression", "(Ljava/lang/String;)V", e) class SetupGenerator(OptionHandler): """ Allows generation of large number of setups using parameter setups. """ def __init__(self, jobject=None, options=None): """ Initializes the specified classifier using its classname or the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ classname = "weka.core.SetupGenerator" if jobject is None: jobject = OptionHandler.new_instance(classname) else: self.enforce_type(jobject, classname) super(SetupGenerator, self).__init__(jobject=jobject, options=options) @property def base_object(self): """ Returns the base object to apply the setups to. :return: the base object :rtype: JavaObject or OptionHandler """ jobj = javabridge.call(self.jobject, "getBaseObject", "()Ljava/io/Serializable;") if OptionHandler.check_type(jobj, "weka.core.OptionHandler"): return OptionHandler(jobj) else: return JavaObject(jobj) @base_object.setter def base_object(self, obj): """ Sets the base object to apply the setups to. :param obj: the object to use (must be serializable!) :type obj: JavaObject """ if not obj.is_serializable: raise Exception("Base object must be serializable: " + obj.classname) javabridge.call(self.jobject, "setBaseObject", "(Ljava/io/Serializable;)V", obj.jobject) @property def parameters(self): """ Returns the list of currently set search parameters. :return: the list of AbstractSearchParameter objects :rtype: list """ array = JavaArray(javabridge.call(self.jobject, "getParameters", "()[Lweka/core/setupgenerator/AbstractParameter;")) result = [] for item in array: result.append(AbstractParameter(jobject=item.jobject)) return result @parameters.setter def parameters(self, params): """ Sets the list of search parameters to use. :param params: list of AbstractSearchParameter objects :type params: list """ array = JavaArray(jobject=JavaArray.new_instance("weka.core.setupgenerator.AbstractParameter", len(params))) for idx, obj in enumerate(params): array[idx] = obj.jobject javabridge.call(self.jobject, "setParameters", "([Lweka/core/setupgenerator/AbstractParameter;)V", array.jobject) def setups(self): """ Generates and returns all the setups according to the parameter search space. :return: the list of configured objects (of type JavaObject) :rtype: list """ result = [] has_options = self.base_object.is_optionhandler enm = javabridge.get_enumeration_wrapper(javabridge.call(self.jobject, "setups", "()Ljava/util/Enumeration;")) while enm.hasMoreElements(): if has_options: result.append(OptionHandler(enm.nextElement())) else: result.append(JavaObject(enm.nextElement())) return result def main(): """ Runs a classifier from the command-line. Calls JVM start/stop automatically. Use -h to see all options. """ parser = argparse.ArgumentParser( description='Performs option handling operations from the command-line. Calls JVM start/stop automatically.') parser.add_argument("-j", metavar="classpath", dest="classpath", help="additional classpath, jars/directories") parser.add_argument("-action", metavar="action", dest="action", required=True, help="The action to perform on the options: join=create single string, "\ "split=create array from quoted option, code=generate code from options") parser.add_argument("option", nargs=argparse.REMAINDER, help="The option(s) to process") parsed = parser.parse_args() jars = [] if parsed.classpath is not None: jars = parsed.classpath.split(os.pathsep) jvm.start(jars, packages=True) try: if parsed.action == "join": output = "cmdline = \"" + backquote(join_options(parsed.option)) + "\"" elif parsed.action == "split" and len(parsed.option) == 1: output = "options = [\n" opts = split_options(parsed.option[0]) for idx, opt in enumerate(opts): if idx > 0: output += ",\n" output += " \"" + backquote(opt) + "\"" output += "]" elif parsed.action == "code": options = parsed.option[:] cname = None # classname + options? if options[0].find(".") > -1: cname = options[0] options = options[1:] output = "options = [ \n" for idx, opt in enumerate(options): if idx > 0: output += ",\n" output += " \"" + backquote(opt) + "\"" output += "]\n" if cname is not None: output += 'handler = OptionHandler(JavaObject.new_instance("' + cname + '"))\n' output += 'handler.options = options\n' else: raise Exception("Unsupported action: " + parsed.action) if output is not None: print(output) except Exception as e: print(e) finally: jvm.stop() if __name__ == "__main__": try: main() except Exception as ex: print(ex) python-weka-wrapper3-0.1.3/python/weka/core/capabilities.py0000644017267001726700000003567412712225064025375 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # capabilities.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge from weka.core.classes import JavaObject, Enum from weka.core.dataset import Attribute, Instances class Capability(Enum): """ Wrapper for a Capability. """ def __init__(self, jobject=None, member=None): """ Initializes the wrapper with the specified Capability object. :param jobject: the Capability object to wrap :type jobject: JB_Object :param member: the capability string to parse :type member: str """ super(Capability, self).__init__(\ jobject=jobject, enum="weka.core.Capabilities$Capability", member=member) @property def is_attribute(self): """ Returns whether this capability is an attribute. :return: whether it is an attribute :rtype: bool """ return javabridge.call(self.jobject, "isAttribute", "()Z") @property def is_attribute_capability(self): """ Returns whether this capability is an attribute capability. :return: whether it is an attribute capability :rtype: bool """ return javabridge.call(self.jobject, "isAttributeCapability", "()Z") @property def is_class(self): """ Returns whether this capability is a class. :return: whether it is a class :rtype: bool """ return javabridge.call(self.jobject, "isClass", "()Z") @property def is_class_capability(self): """ Returns whether this capability is a class capability. :return: whether it is a class capability :rtype: bool """ return javabridge.call(self.jobject, "isClassCapability", "()Z") @property def is_other_capability(self): """ Returns whether this capability is an other capability. :return: whether it is an other capability :rtype: bool """ return javabridge.call(self.jobject, "isOtherCapability", "()Z") class Capabilities(JavaObject): """ Wrapper for Capabilities. """ def __init__(self, jobject=None, owner=None): """ Initializes the wrapper with the specified Capabilities object. :param jobject: the Capabilities object to wrap, instantiates a new one if None :type jobject: JB_Object :param owner: the weka.core.CapabilitiesHandler object to use as owner, can be None :type owner: JB_Object or JavaObject """ if jobject is None: if isinstance(owner, JavaObject): owner = owner.jobject jobject = javabridge.make_instance( "weka/core/Capabilities", "(Lweka/core/CapabilitiesHandler;)V", owner) Capabilities.enforce_type(jobject, "weka.core.Capabilities") super(Capabilities, self).__init__(jobject) @property def owner(self): """ Returns the owner of these capabilities, if any. :return: the owner, can be None :rtype: JavaObject """ obj = javabridge.call(self.jobject, "getOwner", "()Lweka/core/CapabilitiesHandler;") if obj is None: return None else: return JavaObject(jobject=obj) @owner.setter def owner(self, obj): """ Sets the new owner. :param obj: the new owner :type obj: JavaObject or JB_Object """ if isinstance(obj, JavaObject): obj = obj.jobject javabridge.call(self.jobject, "setOwner", "(Lweka/core/CapabilitiesHandler;)V", obj) def capabilities(self): """ Returns all the capabilities. :return: all capabilities :rtype: list """ result = [] iterator = javabridge.iterate_java(javabridge.call(self.jobject, "capabilities", "()Ljava/util/Iterator;")) for c in iterator: result.append(Capability(c)) return result def attribute_capabilities(self): """ Returns all the attribute capabilities. :return: attribute capabilities :rtype: Capabilities """ return Capabilities( javabridge.call(self.jobject, "getAttributeCapabilities", "()Lweka/core/Capabilities;")) def class_capabilities(self): """ Returns all the class capabilities. :return: class capabilities :rtype: Capabilities """ return Capabilities( javabridge.call(self.jobject, "getClassCapabilities", "()Lweka/core/Capabilities;")) def other_capabilities(self): """ Returns all other capabilities. :return: all other capabilities :rtype: Capabilities """ return Capabilities( javabridge.call(self.jobject, "getOtherCapabilities", "()Lweka/core/Capabilities;")) def dependencies(self): """ Returns all the dependencies. :return: the dependency list :rtype: list """ result = [] iterator = javabridge.iterate_java(javabridge.call(self.jobject, "dependencies", "()Ljava/util/Iterator;")) for c in iterator: result.append(Capability(c)) return result def handles(self, capability): """ Returns whether the specified capability is set. :param capability: the capability to check :type capability: Capability :return: whether the capability is set :rtype: bool """ return javabridge.call( self.jobject, "handles", "(Lweka/core/Capabilities$Capability;)Z", capability.jobject) def enable_all(self): """ enables all capabilities. """ javabridge.call(self.jobject, "enableAll", "()V") def enable_all_attributes(self): """ enables all attributes. """ javabridge.call(self.jobject, "enableAllAttributes", "()V") def enable_all_classes(self): """ enables all classes. """ javabridge.call(self.jobject, "enableAllClasses", "()V") def enable(self, capability): """ enables the specified capability. :param capability: the capability to enable :type capability: Capability """ javabridge.call(self.jobject, "enable", "(Lweka/core/Capabilities$Capability;)V", capability.jobject) def enable_all_attribute_dependencies(self): """ enables all attribute dependencies. """ javabridge.call(self.jobject, "enableAllAttributeDependencies", "()V") def enable_all_class_dependencies(self): """ enables all class dependencies. """ javabridge.call(self.jobject, "enableAllClassDependencies", "()V") def enable_dependency(self, capability): """ enables the dependency of the given capability enabling NOMINAL_ATTRIBUTES also enables BINARY_ATTRIBUTES, UNARY_ATTRIBUTES and EMPTY_NOMINAL_ATTRIBUTES. :param capability: the dependency to enable :type capability: Capability """ javabridge.call( self.jobject, "enableDependency", "(Lweka/core/Capabilities$Capability;)V", capability.jobject) def disable_all(self): """ Disables all capabilities. """ javabridge.call(self.jobject, "disableAll", "()V") def disable_all_attributes(self): """ Disables all attributes. """ javabridge.call(self.jobject, "disableAllAttributes", "()V") def disable_all_classes(self): """ Disables all classes. """ javabridge.call(self.jobject, "disableAllClasses", "()V") def disable(self, capability): """ Disables the specified capability. :param capability: the capability to disable :type capability: Capability """ javabridge.call(self.jobject, "disable", "(Lweka/core/Capabilities$Capability;)V", capability.jobject) def disable_all_attribute_dependencies(self): """ Disables all attribute dependencies. """ javabridge.call(self.jobject, "disableAllAttributeDependencies", "()V") def disable_all_class_dependencies(self): """ Disables all class dependencies. """ javabridge.call(self.jobject, "disableAllClassDependencies", "()V") def disable_dependency(self, capability): """ Disables the dependency of the given capability Disabling NOMINAL_ATTRIBUTES also disables BINARY_ATTRIBUTES, UNARY_ATTRIBUTES and EMPTY_NOMINAL_ATTRIBUTES. :param capability: the dependency to disable :type capability: Capability """ javabridge.call( self.jobject, "disableDependency", "(Lweka/core/Capabilities$Capability;)V", capability.jobject) def has_dependencies(self): """ Returns whether any dependencies are set. :return: whether any dependecies are set :rtype: bool """ return javabridge.call(self.jobject, "hasDependencies", "()Z") def has_dependency(self, capability): """ Returns whether the specified dependency is set. :param capability: the capability to check :type capability: Capability :return: whether the dependency is set :rtype: bool """ return javabridge.call( self.jobject, "hasDependency", "(Lweka/core/Capabilities$Capability;)Z", capability.jobject) def supports(self, capabilities): """ Returns true if the currently set capabilities support at least all of the capabiliites of the given Capabilities object (checks only the enum!) :param capabilities: the capabilities to check :type capabilities: Capabilities :return: whether the current capabilities support at least the specified ones :rtype: bool """ return javabridge.call(self.jobject, "supports", "(Lweka/core/Capabilities;)Z", capabilities.jobject) def supports_maybe(self, capabilities): """ Returns true if the currently set capabilities support (or have a dependency) at least all of the capabilities of the given Capabilities object (checks only the enum!) :param capabilities: the capabilities to check :type capabilities: Capabilities :return: whether the current capabilities (potentially) support the specified ones :rtype: bool """ return javabridge.call(self.jobject, "supportsMaybe", "(Lweka/core/Capabilities;)Z", capabilities.jobject) @property def min_instances(self): """ Returns the minimum number of instances that must be supported. :return: the minimum number :rtype: int """ return javabridge.call(self.jobject, "getMinimumNumberInstances", "()I") @min_instances.setter def min_instances(self, minimum): """ Sets the minimum number of instances that must be supported. :param minimum: the minimum number :type minimum: int """ javabridge.call(self.jobject, "setMinimumNumberInstances", "(I)V", minimum) def test_attribute(self, att, is_class=None, fail=False): """ Tests whether the attribute meets the conditions. :param att: the Attribute to test :type att: Attribute :param is_class: whether this attribute is the class attribute :type is_class: bool :param fail: whether to fail with an exception in case the test fails :type fail: bool :return: whether the attribute meets the conditions :rtype: bool """ if fail: if is_class is None: return javabridge.call( self.jobject, "test", "(Lweka/core/Attribute;)Z", att.jobject) else: return javabridge.call( self.jobject, "test", "(Lweka/core/Attribute;Z)Z", att.jobject, is_class) else: if is_class is None: return javabridge.call( self.jobject, "testWithFail", "(Lweka/core/Attribute;)Z", att.jobject) else: return javabridge.call( self.jobject, "testWithFail", "(Lweka/core/Attribute;Z)Z", att.jobject, is_class) def test_instances(self, data, from_index=None, to_index=None, fail=False): """ Tests whether the dataset meets the conditions. :param data: the Instances to test :type data: Instances :param from_index: the first attribute to include :type from_index: int :param to_index: the last attribute to include :type to_index: int :return: wether the dataset meets the requirements :rtype: bool """ if fail: if (from_index is None) or (to_index is None): return javabridge.call( self.jobject, "test", "(Lweka/core/Instances;)Z", data.jobject) else: return javabridge.call( self.jobject, "test", "(Lweka/core/Instances;II)Z", data.jobject, from_index, to_index) else: if (from_index is None) or (to_index is None): return javabridge.call( self.jobject, "testWithFail", "(Lweka/core/Instances;)Z", data.jobject) else: return javabridge.call( self.jobject, "testWithFail", "(Lweka/core/Instances;II)Z", data.jobject, from_index, to_index) @classmethod def for_instances(cls, data, multi=None): """ returns a Capabilities object specific for this data. The minimum number of instances is not set, the check for multi-instance data is optional. :param data: the data to generate the capabilities for :type data: Instances :param multi: whether to check the structure, too :type multi: bool :return: the generated capabilities :rtype: Capabilities """ if multi is None: return Capabilities(javabridge.static_call( "weka/core/Capabilities", "forInstances", "(Lweka/core/Instances;)Lweka/core/Capabilities;", data.jobject)) else: return Capabilities(javabridge.static_call( "weka/core/Capabilities", "forInstances", "(Lweka/core/Instances;Z)Lweka/core/Capabilities;", data.jobject, multi)) python-weka-wrapper3-0.1.3/python/weka/core/tokenizers.py0000644017267001726700000000534012712232237025124 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # tokenizers.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge from weka.core.classes import OptionHandler class TokenIterator(object): """ Iterator for string tokens. """ def __init__(self, tokenizer): """ Initializes the iterator. :param tokenizer: the tokenizer instance to use :type tokenizer: Tokenizer """ self.tokenizer = tokenizer self.__has_more = javabridge.make_call(self.tokenizer.jobject, "hasMoreElements", "()Z") self.__next = javabridge.make_call(self.tokenizer.jobject, "nextElement", "()Ljava/lang/String;") def __iter__(self): """ Returns itself. """ return self def __next__(self): """ Reads the next dataset row. :return: the next row :rtype: Instance """ if not self.__has_more(): raise StopIteration() else: return javabridge.get_env().get_string(self.__next()) class Tokenizer(OptionHandler): """ Wrapper class for tokenizers. """ def __init__(self, classname="weka.core.tokenizers.AlphabeticTokenizer", jobject=None, options=None): """ Initializes the specified tokenizer using either the classname or the supplied JB_Object. :param classname: the classname of the tokenizer :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Tokenizer.new_instance(classname) self.enforce_type(jobject, "weka.core.tokenizers.Tokenizer") super(Tokenizer, self).__init__(jobject=jobject, options=options) def tokenize(self, s): """ Tokenizes the string. :param s: the string to tokenize :type s: str :return: the iterator :rtype: TokenIterator """ javabridge.call(self.jobject, "tokenize", "(Ljava/lang/String;)V", s) return TokenIterator(self) python-weka-wrapper3-0.1.3/python/weka/core/packages.py0000644017267001726700000002522312712225077024513 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # packages.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import weka.core.jvm as jvm from weka.core.classes import JavaObject class Package(JavaObject): """ Wrapper for the weka.core.packageManagement.Package class. """ def __init__(self, jobject): """ Initializes the wrapper. :param jobject: the java Package instance to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.core.packageManagement.Package") super(Package, self).__init__(jobject) @property def name(self): """ Returns the name of the package. :return: the name :rtype: str """ return javabridge.call(self.jobject, "getName", "()Ljava/lang/String;") @property def url(self): """ Returns the URL of the package. :return: the url :rtype: str """ return str(JavaObject(javabridge.call(self.jobject, "getPackageURL", "()Ljava/net/URL;"))) @property def dependencies(self): """ Returns the dependencies of the package. :return: the list of Dependency objects :rtype: list of Dependency """ result = [] dependencies = javabridge.get_collection_wrapper( javabridge.call(self.jobject, "getDependencies", "()Ljava/util/List;")) for dependency in dependencies: result.append(Dependency(dependency)) return result @property def metadata(self): """ Returns the meta-data. :return: the meta-data dictionary :rtype: dict """ return javabridge.get_dictionary_wrapper( javabridge.call(self.jobject, "getPackageMetaData", "()Ljava/util/Map;")) @property def is_installed(self): """ Returns whether the package is installed. :return: whether installed :rtype: bool """ return javabridge.call(self.jobject, "isInstalled", "()Z") def install(self): """ Installs the package. """ return javabridge.call(self.jobject, "install", "()V") class PackageConstraint(JavaObject): """ Wrapper for the weka.core.packageManagement.PackageConstraint class. """ def __init__(self, jobject): """ Initializes the wrapper. :param jobject: the java PackageConstraint instance to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.core.packageManagement.PackageConstraint") super(PackageConstraint, self).__init__(jobject) def set_package(self, pkge): """ Sets the package. :param pkge: the package :type pkge: Package """ javabridge.call(self.jobject, "setPackage", "(Lweka/core/packageManagement/Package;)V", pkge.jobject) def get_package(self): """ Returns the package. :return: the package :rtype: Package """ return Package(javabridge.call(self.jobject, "getPackage", "()Lweka/core/packageManagement/Package;")) def check_constraint(self, pkge=None, constr=None): """ Checks the constraints. :param pkge: the package to check :type pkge: Package :param constr: the package constraint to check :type constr: PackageConstraint """ if not pkge is None: return javabridge.call( self.jobject, "checkConstraint", "(Lweka/core/packageManagement/Package;)Z", pkge.jobject) if not constr is None: return javabridge.call( self.jobject, "checkConstraint", "(Lweka/core/packageManagement/PackageConstraint;)Z", pkge.jobject) raise Exception("Either package or package constraing must be provided!") class Dependency(JavaObject): """ Wrapper for the weka.core.packageManagement.Dependency class. """ def __init__(self, jobject): """ Initializes the wrapper. :param jobject: the java Dependency instance to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.core.packageManagement.Dependency") super(Dependency, self).__init__(jobject) @property def source(self): """ Returns the source package. :return: the package :rtype: Package """ return Package( javabridge.call(self.jobject, "getSource", "()Lweka/core/packageManagement/Package;")) @source.setter def source(self, pkge): """ Sets the source package. :param pkge: the package :type pkge: Package """ javabridge.call( self.jobject, "setSource", "(Lweka/core/packageManagement/Package;)V", pkge.jobject) @property def target(self): """ Returns the target package constraint. :return: the package constraint :rtype: PackageConstraint """ return PackageConstraint( javabridge.call(self.jobject, "getTarget", "()Lweka/core/packageManagement/PackageConstraint;")) @target.setter def target(self, constr): """ Sets the target package constraint. :param constr: the package constraint :type constr: Package """ javabridge.call( self.jobject, "setTarget", "(Lweka/core/packageManagement/PackageConstraint;)V", constr.jobject) def establish_cache(): """ Establishes the package cache if necessary. """ javabridge.static_call( "weka/core/WekaPackageManager", "establishCacheIfNeeded", "([Ljava/io/PrintStream;)Ljava/lang/Exception;", []) def refresh_cache(): """ Refreshes the cache. """ establish_cache() javabridge.static_call( "weka/core/WekaPackageManager", "refreshCache", "([Ljava/io/PrintStream;)Ljava/lang/Exception;", []) def all_packages(): """ Returns a list of all packages. :return: the list of packages :rtype: list """ establish_cache() result = [] pkgs = javabridge.get_collection_wrapper( javabridge.static_call( "weka/core/WekaPackageManager", "getAllPackages", "()Ljava/util/List;")) for pkge in pkgs: result.append(Package(pkge)) return result def available_packages(): """ Returns a list of all packages that aren't installed yet. :return: the list of packages :rtype: list """ establish_cache() result = [] pkgs = javabridge.get_collection_wrapper( javabridge.static_call( "weka/core/WekaPackageManager", "getAvailablePackages", "()Ljava/util/List;")) for pkge in pkgs: result.append(Package(pkge)) return result def installed_packages(): """ Returns a list of the installed packages. :return: the list of packages :rtype: list """ establish_cache() result = [] pkgs = javabridge.get_collection_wrapper( javabridge.static_call( "weka/core/WekaPackageManager", "getInstalledPackages", "()Ljava/util/List;")) for pkge in pkgs: result.append(Package(pkge)) return result def install_package(pkge, version="Latest"): """ The list of packages to install. :param pkge: the name of the repository package, a URL (http/https) or a zip file :type pkge: str :param version: in case of the repository packages, the version :type version: str :return: whether successfully installed :rtype: bool """ establish_cache() if pkge.startswith("http://") or pkge.startswith("https://"): url = javabridge.make_instance( "java/net/URL", "(Ljava/lang/String;)V", javabridge.get_env().new_string_utf(pkge)) return not javabridge.static_call( "weka/core/WekaPackageManager", "installPackageFromURL", "(Ljava/net/URL;[Ljava/io/PrintStream;)Ljava/lang/String;", url, []) is None elif pkge.lower().endswith(".zip"): return not javabridge.static_call( "weka/core/WekaPackageManager", "installPackageFromArchive", "(Ljava/lang/String;[Ljava/io/PrintStream;)Ljava/lang/String;", pkge, []) is None else: return javabridge.static_call( "weka/core/WekaPackageManager", "installPackageFromRepository", "(Ljava/lang/String;Ljava/lang/String;[Ljava/io/PrintStream;)Z", pkge, version, []) def uninstall_package(name): """ Uninstalls a package. :param name: the name of the package :type name: str :return: whether successfully uninstalled :rtype: bool """ establish_cache() javabridge.static_call( "weka/core/WekaPackageManager", "uninstallPackage", "(Ljava/lang/String;Z[Ljava/io/PrintStream;)V", name, True, []) def is_installed(name): """ Checks whether a package with the name is already installed. :param name: the name of the package :type name: str :return: whether the package is installed :rtype: bool """ pkgs = installed_packages() for pkge in pkgs: if pkge.name == name: return True return False if __name__ == "__main__": jvm.start() try: print("Establish cache") print("===============") establish_cache() print("Refresh cache") print("=============") refresh_cache() print("All packages") print("============") packages = all_packages() for pkg in packages: print(pkg.name) print(" url: " + pkg.url) print("") print("Available packages") print("==================") packages = available_packages() p = packages[0] for pkg in packages: print(pkg.name) print(" url: " + pkg.url) print("") print("Installed packages") print("==================") packages = installed_packages() for pkg in packages: print(pkg.name) print(" url: " + pkg.url) print("") except Exception as e: print(e) finally: jvm.stop() python-weka-wrapper3-0.1.3/python/weka/core/__init__.py0000644017267001726700000000152712712225061024466 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # __init__.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) # check whether scipy is there scipy_available = False try: import scipy scipy_available = True except ImportError: pass python-weka-wrapper3-0.1.3/python/weka/core/jvm.py0000644017267001726700000001104413124550577023531 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # jvm.py # Copyright (C) 2014-2017 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import os import glob import logging started = None # logging setup logging.basicConfig(level=logging.DEBUG) logger = logging.getLogger(__name__) def add_bundled_jars(): """ Adds the bundled jars to the JVM's classpath. """ # determine lib directory with jars rootdir = os.path.split(os.path.dirname(__file__))[0] libdir = rootdir + os.sep + "lib" # add jars from lib directory for l in glob.glob(libdir + os.sep + "*.jar"): if l.lower().find("-src.") == -1: javabridge.JARS.append(str(l)) def add_weka_packages(alt_path = None): """ Adds the jars from all Weka packages to the JVM's classpath. :param alt_path: an alternative weka home path :type alt_path: str """ if alt_path is None: prefix = os.getenv("WEKA_HOME", "~" + os.sep + "wekafiles") else: prefix = alt_path package_dir = os.path.expanduser(prefix + os.sep + "packages") if not os.path.exists(package_dir): logger.error("package_dir not found: " + package_dir) return else: logger.debug("package_dir=" + package_dir) # traverse packages for p in os.listdir(package_dir): if os.path.isdir(package_dir + os.sep + p): directory = package_dir + os.sep + p logger.debug(" directory=" + directory) # inspect package directory for l in os.listdir(directory): if l.lower().endswith(".jar"): javabridge.JARS.append(directory + os.sep + l) if l == "lib": for m in glob.glob(directory + os.sep + "lib" + os.sep + "*.jar"): javabridge.JARS.append(m) def add_system_classpath(): """ Adds the system's classpath to the JVM's classpath. """ if 'CLASSPATH' in os.environ: parts = os.environ['CLASSPATH'].split(os.pathsep) for part in parts: javabridge.JARS.append(part) def start(class_path=None, bundled=True, packages=False, system_cp=False, max_heap_size=None): """ Initializes the javabridge connection (starts up the JVM). :param class_path: the additional classpath elements to add :type class_path: list :param bundled: whether to add jars from the "lib" directory :type bundled: bool :param packages: whether to add jars from Weka packages as well (bool) or an alternative Weka home directory (str) :type packages: bool or str :param system_cp: whether to add the system classpath as well :type system_cp: bool :param max_heap_size: the maximum heap size (-Xmx parameter, eg 512m or 4g) :type max_heap_size: str """ global started if started is not None: logger.info("JVM already running, call jvm.stop() first") return # add user-defined jars first if class_path is not None: for cp in class_path: logger.debug("Adding user-supplied classpath=" + cp) javabridge.JARS.append(cp) if bundled: logger.debug("Adding bundled jars") add_bundled_jars() if packages is not None: if isinstance(packages, bool): if packages: logger.debug("Adding Weka packages") add_weka_packages() else: logger.debug("Adding Weka packages, using: " + packages) add_weka_packages(packages) if system_cp: logger.debug("Adding system classpath") add_system_classpath() logger.debug("Classpath=" + str(javabridge.JARS)) logger.debug("MaxHeapSize=" + ("default" if (max_heap_size is None) else max_heap_size)) javabridge.start_vm(run_headless=True, max_heap_size=max_heap_size) javabridge.attach() started = True def stop(): """ Kills the JVM. """ global started if started is not None: started = None javabridge.kill_vm() python-weka-wrapper3-0.1.3/python/weka/core/converters.py0000644017267001726700000003003112712232300025103 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # converters.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge from weka.core.classes import OptionHandler from weka.core.capabilities import Capabilities from weka.core.dataset import Instances, Instance, Attribute import numpy import os class Loader(OptionHandler): """ Wrapper class for Loaders. """ def __init__(self, classname="weka.core.converters.ArffLoader", jobject=None, options=None): """ Initializes the specified loader either using the classname or the JB_Object. :param classname: the classname of the loader :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Loader.new_instance(classname) self.enforce_type(jobject, "weka.core.converters.Loader") super(Loader, self).__init__(jobject=jobject, options=options) self.incremental = False self.structure = None def __iter__(self): """ Returns an iterator in case the loader was instantiated in incremental mode, otherwise an Exception is raised. :return: the iterator :rtype: IncrementalLoaderIterator """ if not self.incremental: raise Exception("Not in incremental mode, cannot iterate!") return IncrementalLoaderIterator(self, self.structure) def load_file(self, dfile, incremental=False): """ Loads the specified file and returns the Instances object. In case of incremental loading, only the structure. :param dfile: the file to load :type dfile: str :param incremental: whether to load the dataset incrementally :type incremental: bool :return: the full dataset or the header (if incremental) :rtype: Instances :raises Exception: if the file does not exist """ self.enforce_type(self.jobject, "weka.core.converters.FileSourcedConverter") self.incremental = incremental if not javabridge.is_instance_of(dfile, "Ljava/io/File;"): dfile = javabridge.make_instance( "Ljava/io/File;", "(Ljava/lang/String;)V", javabridge.get_env().new_string_utf(str(dfile))) javabridge.call(self.jobject, "reset", "()V") # check whether file exists, otherwise previously set file gets loaded again sfile = javabridge.to_string(dfile) if not os.path.exists(sfile): raise Exception("Dataset file does not exist: " + str(sfile)) javabridge.call(self.jobject, "setFile", "(Ljava/io/File;)V", dfile) if incremental: self.structure = Instances(javabridge.call(self.jobject, "getStructure", "()Lweka/core/Instances;")) return self.structure else: return Instances(javabridge.call(self.jobject, "getDataSet", "()Lweka/core/Instances;")) def load_url(self, url, incremental=False): """ Loads the specified URL and returns the Instances object. In case of incremental loading, only the structure. :param url: the URL to load the data from :type url: str :param incremental: whether to load the dataset incrementally :type incremental: bool :return: the full dataset or the header (if incremental) :rtype: Instances """ self.enforce_type(self.jobject, "weka.core.converters.URLSourcedLoader") self.incremental = incremental javabridge.call(self.jobject, "reset", "()V") javabridge.call(self.jobject, "setURL", "(Ljava/lang/String;)V", str(url)) if incremental: self.structure = Instances(javabridge.call(self.jobject, "getStructure", "()Lweka/core/Instances;")) return self.structure else: return Instances(javabridge.call(self.jobject, "getDataSet", "()Lweka/core/Instances;")) class IncrementalLoaderIterator(object): """ Iterator for dataset rows when loarding incrementally. """ def __init__(self, loader, structure): """ :param loader: the loader instance to use for loading the data incrementally :type loader: Loader :param structure: the dataset structure :type structure: Instances """ self.loader = loader self.structure = structure def __iter__(self): """ Returns itself. """ return self def __next__(self): """ Reads the next dataset row. :return: the next row :rtype: Instance """ result = javabridge.call( self.loader.jobject, "getNextInstance", "(Lweka/core/Instances;)Lweka/core/Instance;", self.structure.jobject) if result is None: raise StopIteration() else: return Instance(result) class TextDirectoryLoader(OptionHandler): """ Wrapper class for TextDirectoryLoader. """ def __init__(self, jobject=None, options=None): """ Initializes the text directory loader either using a new instance or the JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = TextDirectoryLoader.new_instance("weka.core.converters.TextDirectoryLoader") self.enforce_type(jobject, "weka.core.converters.TextDirectoryLoader") super(TextDirectoryLoader, self).__init__(jobject=jobject, options=options) def load(self): """ Loads the text files from the specified directory and returns the Instances object. In case of incremental loading, only the structure. :return: the full dataset or the header (if incremental) :rtype: Instances """ javabridge.call(self.jobject, "reset", "()V") return Instances(javabridge.call(self.jobject, "getDataSet", "()Lweka/core/Instances;")) class Saver(OptionHandler): """ Wrapper class for Savers. """ def __init__(self, classname="weka.core.converters.ArffSaver", jobject=None, options=None): """ Initializes the specified saver either using the classname or the provided JB_Object. :param classname: the classname of the saver :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to use :type options: list """ if jobject is None: jobject = Saver.new_instance(classname) self.enforce_type(jobject, "weka.core.converters.Saver") super(Saver, self).__init__(jobject=jobject, options=options) def capabilities(self): """ Returns the capabilities of the saver. :return: the capabilities :rtype: Capabilities """ return Capabilities(javabridge.call(self.jobject, "getCapabilities", "()Lweka/core/Capabilities;")) def save_file(self, data, dfile): """ Saves the Instances object in the specified file. :param data: the data to save :type data: Instances :param dfile: the file to save the data to :type dfile: str """ self.enforce_type(self.jobject, "weka.core.converters.FileSourcedConverter") if not javabridge.is_instance_of(dfile, "Ljava/io/File;"): dfile = javabridge.make_instance( "Ljava/io/File;", "(Ljava/lang/String;)V", javabridge.get_env().new_string_utf(str(dfile))) javabridge.call(self.jobject, "setFile", "(Ljava/io/File;)V", dfile) javabridge.call(self.jobject, "setInstances", "(Lweka/core/Instances;)V", data.jobject) javabridge.call(self.jobject, "writeBatch", "()V") def loader_for_file(filename): """ Returns a Loader that can load the specified file, based on the file extension. None if failed to determine. :param filename: the filename to get the loader for :type filename: str :return: the assoicated loader instance or None if none found :rtype: Loader """ loader = javabridge.static_call( "weka/core/converters/ConverterUtils", "getLoaderForFile", "(Ljava/lang/String;)Lweka/core/converters/AbstractFileLoader;", filename) if loader is None: return None else: return Loader(jobject=loader) def load_any_file(filename): """ Determines a Loader based on the the file extension. If successful, loads the full dataset and returns it. :param filename: the name of the file to load :type filename: str :return: the :rtype: Instances """ loader = loader_for_file(filename) if loader is None: return None else: return loader.load_file(filename) def saver_for_file(filename): """ Returns a Saver that can load the specified file, based on the file extension. None if failed to determine. :param filename: the filename to get the saver for :type filename: str :return: the associated saver instance or None if none found :rtype: Saver """ saver = javabridge.static_call( "weka/core/converters/ConverterUtils", "getSaverForFile", "(Ljava/lang/String;)Lweka/core/converters/AbstractFileSaver;", filename) if saver is None: return None else: return Saver(jobject=saver) def save_any_file(data, filename): """ Determines a Saver based on the the file extension. Returns whether successfully saved. :param filename: the name of the file to save :type filename: str :param data: the data to save :type data: Instances :return: whether successfully saved :rtype: bool """ saver = saver_for_file(filename) if saver is None: return False else: saver.save_file(data, filename) return True def ndarray_to_instances(array, relation, att_template="Att-#", att_list=None): """ Converts the numpy matrix into an Instances object and returns it. :param array: the numpy ndarray to convert :type array: numpy.darray :param relation: the name of the dataset :type relation: str :param att_template: the prefix to use for the attribute names, "#" is the 1-based index, "!" is the 0-based index, "@" the relation name :type att_template: str :param att_list: the list of attribute names to use :type att_list: list :return: the generated instances object :rtype: Instances """ if len(numpy.shape(array)) != 2: raise Exception("Number of array dimensions must be 2!") rows, cols = numpy.shape(array) # header atts = [] if att_list is not None: if len(att_list) != cols: raise Exception( "Number columns and provided attribute names differ: " + str(cols) + " != " + len(att_list)) for name in att_list: att = Attribute.create_numeric(name) atts.append(att) else: for i in range(cols): name = att_template.replace("#", str(i+1)).replace("!", str(i)).replace("@", relation) att = Attribute.create_numeric(name) atts.append(att) result = Instances.create_instances(relation, atts, rows) # data for i in range(rows): inst = Instance.create_instance(array[i]) result.add_instance(inst) return result python-weka-wrapper3-0.1.3/python/weka/core/typeconv.py0000644017267001726700000000606013002771014024570 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # typeconv.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import numpy # logging setup logger = logging.getLogger(__name__) def string_array_to_list(a): """ Turns the Java string array into Python unicode string list. :param a: the string array to convert :type a: JB_Object :return: the string list :rtype: list """ result = [] length = javabridge.get_env().get_array_length(a) wrapped = javabridge.get_env().get_object_array_elements(a) for i in range(length): result.append(javabridge.get_env().get_string(wrapped[i])) return result def string_list_to_array(l): """ Turns a Python unicode string list into a Java String array. :param l: the string list :type: list :rtype: java string array :return: JB_Object """ result = javabridge.get_env().make_object_array(len(l), javabridge.get_env().find_class("java/lang/String")) for i in range(len(l)): javabridge.get_env().set_object_array_element(result, i, javabridge.get_env().new_string_utf(l[i])) return result def double_matrix_to_ndarray(m): """ Turns the Java matrix (2-dim array) of doubles into a numpy 2-dim array. :param m: the double matrix :type: JB_Object :return: Numpy array :rtype: numpy.darray """ rows = javabridge.get_env().get_object_array_elements(m) num_rows = len(rows) num_cols = javabridge.get_env().get_array_length(rows[0]) result = numpy.zeros(num_rows * num_cols).reshape((num_rows, num_cols)) i = 0 for row in rows: elements = javabridge.get_env().get_double_array_elements(row) n = 0 for element in elements: result[i][n] = element n += 1 i += 1 return result def enumeration_to_list(enm): """ Turns the java.util.Enumeration into a list. :param enm: the enumeration to convert :type enm: JB_Object :return: the list :rtype: list """ result = [] while javabridge.call(enm, "hasMoreElements", "()Z"): result.append(javabridge.call(enm, "nextElement", "()Ljava/lang/Object;")) return result def double_to_float(d): """ Turns the Python float into a Java java.lang.Float object. :param d: the Python float :type d: float :return: the Float object :rtype: JB_Object """ return javabridge.make_instance("java/lang/Float", "(D)V", d) python-weka-wrapper3-0.1.3/python/weka/core/stemmers.py0000644017267001726700000000365512712225104024570 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # stemmers.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge from weka.core.classes import OptionHandler class Stemmer(OptionHandler): """ Wrapper class for stemmers. """ def __init__(self, classname="weka.core.stemmers.NullStemmer", jobject=None, options=None): """ Initializes the specified stemmer using either the classname or the supplied JB_Object. :param classname: the classname of the stemmer :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Stemmer.new_instance(classname) self.enforce_type(jobject, "weka.core.stemmers.Stemmer") super(Stemmer, self).__init__(jobject=jobject, options=options) self.__stem = javabridge.make_call(self.jobject, "stem", "(Ljava/lang/String;)Ljava/lang/String;") def stem(self, s): """ Performs stemming on the string. :param s: the string to stem :type s: str :return: the stemmed string :rtype: str """ return javabridge.get_env().get_string(self.__stem(javabridge.get_env().new_string_utf(s))) python-weka-wrapper3-0.1.3/python/weka/core/database.py0000644017267001726700000001373212712225074024500 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # database.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge from weka.core.classes import OptionHandler from weka.core.dataset import Instances class DatabaseUtils(OptionHandler): """ Wrapper class for weka.experiment.DatabaseUtils. """ def __init__(self, jobject=None, options=None): """ Initializes a DatabaseUtils object from scratch or uses the provided JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to use :type options: list """ if jobject is None: jobject = DatabaseUtils.new_instance("weka.experiment.DatabaseUtils") self.enforce_type(jobject, "weka.experiment.DatabaseUtils") super(DatabaseUtils, self).__init__(jobject=jobject, options=options) @property def db_url(self): """ Obtains the currently set database URL. :return: the database URL :rtype: str """ return javabridge.call(self.jobject, "getDatabaseURL", "()Ljava/lang/String;") @db_url.setter def db_url(self, url): """ Sets the database URL. :param url: the database URL :type url: str """ javabridge.call(self.jobject, "setDatabaseURL", "(Ljava/lang/String;)V", url) @property def user(self): """ Obtains the currently set database user. :return: the database user :rtype: str """ return javabridge.call(self.jobject, "getUsername", "()Ljava/lang/String;") @user.setter def user(self, user): """ Sets the database user. :param user: the database user :type user: str """ javabridge.call(self.jobject, "setUsername", "(Ljava/lang/String;)V", user) @property def password(self): """ Obtains the currently set database password. :return: the database password :rtype: str """ return javabridge.call(self.jobject, "getPassword", "()Ljava/lang/String;") @password.setter def password(self, password): """ Sets the database password. :param password: the database password :type password: str """ javabridge.call(self.jobject, "setPassword", "(Ljava/lang/String;)V", password) class InstanceQuery(DatabaseUtils): """ Wrapper class for weka.experiment.InstanceQuery. """ def __init__(self, jobject=None, options=None): """ Initializes an InstanceQuery object from scratch or uses the provided JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to use :type options: list """ if jobject is None: jobject = InstanceQuery.new_instance("weka.experiment.InstanceQuery") self.enforce_type(jobject, "weka.experiment.InstanceQuery") super(InstanceQuery, self).__init__(jobject=jobject, options=options) @property def custom_properties(self): """ Obtains the currently set custom properties file. :return: the custom properties file :rtype: str """ return javabridge.to_string(javabridge.call(self.jobject, "getCustomPropsFile", "()Ljava/io/File;")) @custom_properties.setter def custom_properties(self, props): """ Sets the custom properties file to use. :param props: the props file :type props: str """ fprops = javabridge.make_instance("java/io/File", "(Ljava/lang/String;)V", props) javabridge.call(self.jobject, "setCustomPropsFile", "(Ljava/io/File;)V", fprops) @property def sparse_data(self): """ Obtains the whether sparse data is returned or not. :return: whether sparse data is generated :rtype: bool """ return javabridge.call(self.jobject, "getSparseData", "()Z") @sparse_data.setter def sparse_data(self, sparse): """ Sets whether to generate sparse data. :param sparse: whether to generated sparse data :type sparse: bool """ javabridge.call(self.jobject, "setSparseData", "(Z)V", sparse) @property def query(self): """ Obtains the current SQL query to execute. :return: the SQL query :rtype: str """ return javabridge.call(self.jobject, "getQuery", "()Ljava/lang/String;") @query.setter def query(self, query): """ Sets the SQL query to execute. :param query: the SQL query :type query: str """ javabridge.call(self.jobject, "setQuery", "(Ljava/lang/String;)V", query) def retrieve_instances(self, query=None): """ Executes either the supplied query or the one set via options (or the 'query' property). :param query: query to execute if not the currently set one :type query: str :return: the generated dataq :rtype: Instances """ if query is None: data = javabridge.call(self.jobject, "retrieveInstances", "()Lweka/core/Instances;") else: data = javabridge.call(self.jobject, "retrieveInstances", "(Ljava/lang/String;)Lweka/core/Instances;") return Instances(data) python-weka-wrapper3-0.1.3/python/weka/clusterers.py0000644017267001726700000003621012712225043024167 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # clusterers.py # Copyright (C) 2014-201 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import os import sys import argparse import weka.core.jvm as jvm import weka.core.classes as classes from weka.core.classes import JavaObject, join_options from weka.core.classes import OptionHandler from weka.core.classes import Random from weka.core.capabilities import Capabilities from weka.filters import Filter # logging setup logger = logging.getLogger("weka.clusterers") class Clusterer(OptionHandler): """ Wrapper class for clusterers. """ def __init__(self, classname="weka.clusterers.SimpleKMeans", jobject=None, options=None): """ Initializes the specified clusterer using either the classname or the supplied JB_Object. :param classname: the classname of the clusterer :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to use :type options: list """ if jobject is None: jobject = Clusterer.new_instance(classname) self.is_updateable = self.check_type(jobject, "weka.clusterers.UpdateableClusterer") self.is_drawable = self.check_type(jobject, "weka.core.Drawable") self.enforce_type(jobject, "weka.clusterers.Clusterer") super(Clusterer, self).__init__(jobject=jobject, options=options) self.__cluster = javabridge.make_call(self.jobject, "clusterInstance", "(Lweka/core/Instance;)I") self.__distribution = javabridge.make_call(self.jobject, "distributionForInstance", "(Lweka/core/Instance;)[D") @property def capabilities(self): """ Returns the capabilities of the clusterer. :return: the capabilities :rtype: Capabilities """ return Capabilities(javabridge.call(self.jobject, "getCapabilities", "()Lweka/core/Capabilities;")) def build_clusterer(self, data): """ Builds the clusterer with the data. :param data: the data to use for training the clusterer :type data: Instances """ javabridge.call(self.jobject, "buildClusterer", "(Lweka/core/Instances;)V", data.jobject) def update_clusterer(self, inst): """ Updates the clusterer with the instance. :param inst: the Instance to update the clusterer with :type inst: Instance """ if self.is_updateable: javabridge.call(self.jobject, "updateClusterer", "(Lweka/core/Instance;)V", inst.jobject) else: logger.critical(classes.get_classname(self.jobject) + " is not updateable!") def update_finished(self): """ Signals the clusterer that updating with new data has finished. """ if self.is_updateable: javabridge.call(self.jobject, "updateFinished", "()V") else: logger.critical(classes.get_classname(self.jobject) + " is not updateable!") def cluster_instance(self, inst): """ Peforms a prediction. :param inst: the instance to determine the cluster for :type inst: Instance :return: the clustering result :rtype: float """ return self.__cluster(inst.jobject) def distribution_for_instance(self, inst): """ Peforms a prediction, returning the cluster distribution. :param inst: the Instance to get the cluster distribution for :type inst: Instance :return: the cluster distribution :rtype: float[] """ pred = self.__distribution(inst.jobject) return javabridge.get_env().get_double_array_elements(pred) @property def number_of_clusters(self): """ Returns the number of clusters found. :return: the number fo clusters :rtype: int """ return javabridge.call(self.jobject, "numberOfClusters", "()I") @property def graph_type(self): """ Returns the graph type if classifier implements weka.core.Drawable, otherwise -1. :return: the type :rtype: int """ if self.is_drawable: return javabridge.call(self.jobject, "graphType", "()I") else: return -1 @property def graph(self): """ Returns the graph if classifier implements weka.core.Drawable, otherwise None. :return: the graph or None if not available :rtype: str """ if self.is_drawable: return javabridge.call(self.jobject, "graph", "()Ljava/lang/String;") else: return None @classmethod def make_copy(cls, clusterer): """ Creates a copy of the clusterer. :param clusterer: the clustererto copy :type clusterer: Clusterer :return: the copy of the clusterer :rtype: Clusterer """ return Clusterer( jobject=javabridge.static_call( "weka/clusterers/AbstractClusterer", "makeCopy", "(Lweka/clusterers/Clusterer;)Lweka/clusterers/Clusterer;", clusterer.jobject)) class SingleClustererEnhancer(Clusterer): """ Wrapper class for clusterers that use a single base clusterer. """ def __init__(self, classname=None, jobject=None, options=None): """ Initializes the specified clusterer using either the classname or the supplied JB_Object. :param classname: the classname of the clusterer :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to use :type options: list """ if jobject is None: jobject = Clusterer.new_instance(classname) self.enforce_type(jobject, "weka.clusterers.SingleClustererEnhancer") super(SingleClustererEnhancer, self).__init__(classname=classname, jobject=jobject, options=options) @property def clusterer(self): """ Returns the base clusterer. :return: the clusterer :rtype: Clusterer """ return Clusterer(jobject=javabridge.call(self.jobject, "getClusterer", "()Lweka/clusterers/Clusterer;")) @clusterer.setter def clusterer(self, clusterer): """ Sets the base clusterer. :param clusterer: the base clusterer to use :type clusterer: Clusterer """ javabridge.call(self.jobject, "setClusterer", "(Lweka/clusterers/Clusterer;)V", clusterer.jobject) class FilteredClusterer(SingleClustererEnhancer): """ Wrapper class for the filtered clusterer. """ def __init__(self, jobject=None, options=None): """ Initializes the specified clusterer using either the classname or the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to use :type options: list """ classname = "weka.clusterers.FilteredClusterer" if jobject is None: jobject = Clusterer.new_instance(classname) self.enforce_type(jobject, classname) super(FilteredClusterer, self).__init__(classname=classname, jobject=jobject, options=options) @property def filter(self): """ Returns the filter. :return: the filter :rtype: Filter """ return Filter(jobject=javabridge.call(self.jobject, "getFilter", "()Lweka/filters/Filter;")) @filter.setter def filter(self, filtr): """ Sets the filter. :param filtr: the filter to use :type filtr: Filter """ javabridge.call(self.jobject, "setFilter", "(Lweka/filters/Filter;)V", filtr.jobject) class ClusterEvaluation(JavaObject): """ Evaluation class for clusterers. """ def __init__(self): """ Initializes a ClusterEvaluation object. """ super(ClusterEvaluation, self).__init__(ClusterEvaluation.new_instance("weka.clusterers.ClusterEvaluation")) def set_model(self, clusterer): """ Sets the built clusterer to evaluate. :param clusterer: the clusterer to evaluate :type clusterer: Clusterer """ javabridge.call(self.jobject, "setClusterer", "(Lweka/clusterers/Clusterer;)V", clusterer.jobject) def test_model(self, test): """ Evaluates the currently set clusterer on the test set. :param test: the test set to use for evaluating :type test: Instances """ javabridge.call(self.jobject, "evaluateClusterer", "(Lweka/core/Instances;)V", test.jobject) @property def cluster_results(self): """ The cluster results as string. :return: the results string :rtype: str """ return javabridge.call(self.jobject, "clusterResultsToString", "()Ljava/lang/String;") @property def cluster_assignments(self): """ Return an array of cluster assignments corresponding to the most recent set of instances clustered. :return: the cluster assignments :rtype: ndarray """ array = javabridge.call(self.jobject, "getClusterAssignments", "()[D") if array is None: return None else: return javabridge.get_env().get_double_array_elements(array) @property def num_clusters(self): """ Returns the number of clusters. :return: the number of clusters :rtype: int """ return javabridge.call(self.jobject, "getNumClusters", "()I") @property def log_likelihood(self): """ Returns the log likelihood. :return: the log likelihood :rtype: float """ return javabridge.call(self.jobject, "getLogLikelihood", "()D") @property def classes_to_clusters(self): """ Return the array (ordered by cluster number) of minimum error class to cluster mappings. :return: the mappings :rtype: ndarray """ array = javabridge.call(self.jobject, "getClassesToClusters", "()[I") if array is None: return None else: return javabridge.get_env().get_int_array_elements(array) @classmethod def evaluate_clusterer(cls, clusterer, args): """ Evaluates the clusterer with the given options. :param clusterer: the clusterer instance to evaluate :type clusterer: Clusterer :param args: the command-line arguments :type args: list :return: the evaluation result :rtype: str """ return javabridge.static_call( "Lweka/clusterers/ClusterEvaluation;", "evaluateClusterer", "(Lweka/clusterers/Clusterer;[Ljava/lang/String;)Ljava/lang/String;", clusterer.jobject, args) @classmethod def crossvalidate_model(cls, clusterer, data, num_folds, rnd): """ Cross-validates the clusterer and returns the loglikelihood. :param clusterer: the clusterer instance to evaluate :type clusterer: Clusterer :param data: the data to evaluate on :type data: Instances :param num_folds: the number of folds :type num_folds: int :param rnd: the random number generator to use :type rnd: Random :return: the cross-validated loglikelihood :rtype: float """ return javabridge.static_call( "Lweka/clusterers/ClusterEvaluation;", "crossValidateModel", "(Lweka/clusterers/DensityBasedClusterer;Lweka/core/Instances;ILjava/util/Random;)D", clusterer.jobject, data.jobject, num_folds, rnd.jobject) def main(): """ Runs a clusterer from the command-line. Calls JVM start/stop automatically. Use -h to see all options. """ parser = argparse.ArgumentParser( description='Performs clustering from the command-line. Calls JVM start/stop automatically.') parser.add_argument("-j", metavar="classpath", dest="classpath", help="additional classpath, jars/directories") parser.add_argument("-X", metavar="heap", dest="heap", help="max heap size for jvm, e.g., 512m") parser.add_argument("-t", metavar="train", dest="train", required=True, help="training set file") parser.add_argument("-T", metavar="test", dest="test", help="test set file") parser.add_argument("-d", metavar="outmodel", dest="outmodel", help="model output file name") parser.add_argument("-l", metavar="inmodel", dest="inmodel", help="model input file name") parser.add_argument("-p", metavar="attributes", dest="attributes", help="attribute range") parser.add_argument("-x", metavar="num folds", dest="numfolds", help="number of folds") parser.add_argument("-s", metavar="seed", dest="seed", help="seed value for randomization") parser.add_argument("-c", metavar="class index", dest="classindex", help="1-based class attribute index") parser.add_argument("-g", metavar="graph", dest="graph", help="graph output file (if supported)") parser.add_argument("clusterer", help="clusterer classname, e.g., weka.clusterers.SimpleKMeans") parser.add_argument("option", nargs=argparse.REMAINDER, help="additional clusterer options") parsed = parser.parse_args() jars = [] if parsed.classpath is not None: jars = parsed.classpath.split(os.pathsep) params = [] if parsed.train is not None: params.extend(["-t", parsed.train]) if parsed.test is not None: params.extend(["-T", parsed.test]) if parsed.outmodel is not None: params.extend(["-d", parsed.outmodel]) if parsed.inmodel is not None: params.extend(["-l", parsed.inmodel]) if parsed.attributes is not None: params.extend(["-p", parsed.attributes]) if parsed.numfolds is not None: params.extend(["-x", parsed.numfolds]) if parsed.seed is not None: params.extend(["-s", parsed.seed]) if parsed.classindex is not None: params.extend(["-c", parsed.classindex]) if parsed.graph is not None: params.extend(["-g", parsed.graph]) jvm.start(jars, max_heap_size=parsed.heap, packages=True) logger.debug("Commandline: " + join_options(sys.argv[1:])) try: clusterer = Clusterer(classname=parsed.clusterer) if len(parsed.option) > 0: clusterer.options = parsed.option print(ClusterEvaluation.evaluate_clusterer(clusterer, params)) except Exception as e: print(e) finally: jvm.stop() if __name__ == "__main__": try: main() except Exception as ex: print(ex) python-weka-wrapper3-0.1.3/python/weka/attribute_selection.py0000644017267001726700000003052712712227256026061 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # attribute_selection.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import argparse import os import sys import weka.core.jvm as jvm import weka.core.typeconv as typeconv from weka.core.classes import JavaObject, join_options from weka.core.classes import OptionHandler from weka.core.capabilities import Capabilities from weka.core.dataset import Instances, Instance # logging setup logger = logging.getLogger("weka.attribute_selection") class ASSearch(OptionHandler): """ Wrapper class for attribute selection search algorithm. """ def __init__(self, classname="weka.attributeSelection.BestFirst", jobject=None, options=None): """ Initializes the specified search algorithm using either the classname or the supplied JB_Object. :param classname: the classname of the search algorithms :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the commandline options to use :type options: list """ if jobject is None: jobject = ASSearch.new_instance(classname) self.enforce_type(jobject, "weka.attributeSelection.ASSearch") super(ASSearch, self).__init__(jobject=jobject, options=options) def search(self, evaluation, data): """ Performs the search and returns the indices of the selected attributes. :param evaluation: the evaluation algorithm to use :type evaluation: ASEvaluation :param data: the data to use :type data: Instances :return: the selected attributes (0-based indices) :rtype: ndarray """ array = javabridge.call( self.jobject, "search", "(Lweka/attributeSelection/ASEvaluation;Lweka/core/Instances;)[I", evaluation.jobject, data.jobject) if array is None: return None else: javabridge.get_env().get_int_array_elements(array) class ASEvaluation(OptionHandler): """ Wrapper class for attribute selection evaluation algorithm. """ def __init__(self, classname="weka.attributeSelection.CfsSubsetEval", jobject=None, options=None): """ Initializes the specified search algorithm using either the classname or the supplied JB_Object. :param classname: the classname of the search algorithms :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = ASEvaluation.new_instance(classname) self.enforce_type(jobject, "weka.attributeSelection.ASEvaluation") super(ASEvaluation, self).__init__(jobject=jobject, options=options) @property def capabilities(self): """ Returns the capabilities of the classifier. :return: the capabilities :rtype: Capabilities """ return Capabilities(javabridge.call(self.jobject, "getCapabilities", "()Lweka/core/Capabilities;")) def build_evaluator(self, data): """ Builds the evaluator with the data. :param data: the data to use :type data: Instances """ javabridge.call(self.jobject, "buildEvaluator", "(Lweka/core/Instances;)V", data.jobject) def post_process(self, indices): """ Post-processes the evaluator with the selected attribute indices. :param indices: the attribute indices list to use :type indices: ndarray :return: the processed indices :rtype: ndarray """ array = javabridge.call(self.jobject, "postProcess", "([I)[I", indices) if array is None: return None else: return javabridge.get_env().get_int_array_elements(array) class AttributeSelection(JavaObject): """ Performs attribute selection using search and evaluation algorithms. """ def __init__(self): """ Initializes the attribute selection. """ jobject = AttributeSelection.new_instance("weka.attributeSelection.AttributeSelection") super(AttributeSelection, self).__init__(jobject) def evaluator(self, evaluator): """ Sets the evaluator to use. :param evaluator: the evaluator to use. :type evaluator: ASEvaluation """ javabridge.call(self.jobject, "setEvaluator", "(Lweka/attributeSelection/ASEvaluation;)V", evaluator.jobject) def search(self, search): """ Sets the search algorithm to use. :param search: the search algorithm :type search: ASSearch """ javabridge.call(self.jobject, "setSearch", "(Lweka/attributeSelection/ASSearch;)V", search.jobject) def folds(self, folds): """ Sets the number of folds to use for cross-validation. :param folds: the number of folds :type folds: int """ javabridge.call(self.jobject, "setFolds", "(I)V", folds) def ranking(self, ranking): """ Sets whether to perform a ranking, if possible. :param ranking: whether to perform a ranking :type ranking: bool """ javabridge.call(self.jobject, "setRanking", "(Z)V", ranking) def seed(self, seed): """ Sets the seed for cross-validation. :param seed: the seed value :type seed: int """ javabridge.call(self.jobject, "setSeed", "(I)V", seed) def crossvalidation(self, crossvalidation): """ Sets whether to perform cross-validation. :param crossvalidation: whether to perform cross-validation :type crossvalidation: bool """ javabridge.call(self.jobject, "setXval", "(Z)V", crossvalidation) def select_attributes(self, instances): """ Performs attribute selection on the given dataset. :param instances: the data to process :type instances: Instances """ javabridge.call(self.jobject, "SelectAttributes", "(Lweka/core/Instances;)V", instances.jobject) def select_attributes_cv_split(self, instances): """ Performs attribute selection on the given cross-validation split. :param instances: the data to process :type instances: Instances """ javabridge.call(self.jobject, "selectAttributesCVSplit", "(Lweka/core/Instances;)V", instances.jobject) @property def selected_attributes(self): """ Returns the selected attributes from the last run. :return: the Numpy array of 0-based indices :rtype: ndarray """ array = javabridge.call(self.jobject, "selectedAttributes", "()[I") if array is None: return None else: return javabridge.get_env().get_int_array_elements(array) @property def results_string(self): """ Generates a results string from the last attribute selection. :return: the results string :rtype: str """ return javabridge.call(self.jobject, "toResultsString", "()Ljava/lang/String;") @property def cv_results(self): """ Generates a results string from the last cross-validation attribute selection. :return: the results string :rtype: str """ return javabridge.call(self.jobject, "CVResultsString", "()Ljava/lang/String;") @property def number_attributes_selected(self): """ Returns the number of attributes that were selected. :return: the number of attributes :rtype: int """ return javabridge.call(self.jobject, "numberAttributesSelected", "()I") @property def ranked_attributes(self): """ Returns the matrix of ranked attributes from the last run. :return: the Numpy matrix :rtype: ndarray """ matrix = javabridge.call(self.jobject, "rankedAttributes", "()[[D") if matrix is None: return None else: return typeconv.double_matrix_to_ndarray(matrix) def reduce_dimensionality(self, data): """ Reduces the dimensionality of the provided Instance or Instances object. :param data: the data to process :type data: Instances :return: the reduced dataset :rtype: Instances """ if type(data) is Instance: return Instance( javabridge.call( self.jobject, "reduceDimensionality", "(Lweka/core/Instance;)Lweka/core/Instance;", data.jobject)) else: return Instances( javabridge.call( self.jobject, "reduceDimensionality", "(Lweka/core/Instances;)Lweka/core/Instances;", data.jobject)) @classmethod def attribute_selection(cls, evaluator, args): """ Performs attribute selection using the given attribute evaluator and options. :param evaluator: the evaluator to use :type evaluator: ASEvaluation :param args: the command-line args for the attribute selection :type args: list :return: the results string :rtype: str """ return javabridge.static_call( "Lweka/attributeSelection/AttributeSelection;", "SelectAttributes", "(Lweka/attributeSelection/ASEvaluation;[Ljava/lang/String;)Ljava/lang/String;", evaluator.jobject, args) def main(): """ Runs attribute selection from the command-line. Calls JVM start/stop automatically. Use -h to see all options. """ parser = argparse.ArgumentParser( description='Performs attribute selection from the command-line. Calls JVM start/stop automatically.') parser.add_argument("-j", metavar="classpath", dest="classpath", help="additional classpath, jars/directories") parser.add_argument("-X", metavar="heap", dest="heap", help="max heap size for jvm, e.g., 512m") parser.add_argument("-i", metavar="input", dest="input", required=True, help="input file") parser.add_argument("-c", metavar="class index", dest="classindex", help="1-based class attribute index") parser.add_argument("-s", metavar="search", dest="search", help="search method, classname and options") parser.add_argument("-x", metavar="num folds", dest="numfolds", help="number of folds") parser.add_argument("-n", metavar="seed", dest="seed", help="the seed value for randomization") parser.add_argument("evaluator", help="evaluator classname, e.g., weka.attributeSelection.CfsSubsetEval") parser.add_argument("option", nargs=argparse.REMAINDER, help="additional evaluator options") parsed = parser.parse_args() jars = [] if parsed.classpath is not None: jars = parsed.classpath.split(os.pathsep) params = [] if parsed.input is not None: params.extend(["-i", parsed.input]) if parsed.classindex is not None: params.extend(["-c", parsed.classindex]) if parsed.search is not None: params.extend(["-s", parsed.search]) if parsed.numfolds is not None: params.extend(["-x", parsed.numfolds]) if parsed.seed is not None: params.extend(["-n", parsed.seed]) jvm.start(jars, max_heap_size=parsed.heap, packages=True) logger.debug("Commandline: " + join_options(sys.argv[1:])) try: evaluation = ASEvaluation(classname=parsed.evaluator) if len(parsed.option) > 0: evaluation.options = parsed.option print(AttributeSelection.attribute_selection(evaluation, params)) except Exception as e: print(e) finally: jvm.stop() if __name__ == "__main__": try: main() except Exception as ex: print(ex) python-weka-wrapper3-0.1.3/python/weka/classifiers.py0000644017267001726700000021325013107165423024310 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # classifiers.py # Copyright (C) 2014-2017 Fracpete (pythonwekawrapper at gmail dot com) import sys import os import javabridge import logging import argparse import weka.core.jvm as jvm import weka.core.typeconv as typeconv import weka.core.classes as classes from numpy import * from weka.core.classes import JavaObject, join_options, OptionHandler, Random, SelectedTag, Tags, Tag, JavaArray from weka.core.classes import AbstractParameter from weka.core.capabilities import Capabilities from weka.core.dataset import Instances, Instance, Attribute from weka.filters import Filter # logging setup logger = logging.getLogger("weka.classifiers") class Classifier(OptionHandler): """ Wrapper class for classifiers. """ def __init__(self, classname="weka.classifiers.rules.ZeroR", jobject=None, options=None): """ Initializes the specified classifier using either the classname or the supplied JB_Object. :param classname: the classname of the classifier :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Classifier.new_instance(classname) self.enforce_type(jobject, "weka.classifiers.Classifier") self.is_updateable = self.check_type(jobject, "weka.classifiers.UpdateableClassifier") self.is_drawable = self.check_type(jobject, "weka.core.Drawable") self.is_batchpredictor = self.check_type(jobject, "weka.core.BatchPredictor") super(Classifier, self).__init__(jobject=jobject, options=options) self.__classify = javabridge.make_call(self.jobject, "classifyInstance", "(Lweka/core/Instance;)D") self.__distribution = javabridge.make_call(self.jobject, "distributionForInstance", "(Lweka/core/Instance;)[D") if self.is_batchpredictor: self.__distributions = javabridge.make_call( self.jobject, "distributionsForInstances", "(Lweka/core/Instances;)[[D") @property def capabilities(self): """ Returns the capabilities of the classifier. :return: the capabilities :rtype: Capabilities """ return Capabilities(javabridge.call(self.jobject, "getCapabilities", "()Lweka/core/Capabilities;")) def build_classifier(self, data): """ Builds the classifier with the data. :param data: the data to train the classifier with :type data: Instances """ javabridge.call(self.jobject, "buildClassifier", "(Lweka/core/Instances;)V", data.jobject) def update_classifier(self, inst): """ Updates the classifier with the instance. :param inst: the Instance to update the classifier with :type inst: Instance """ if self.is_updateable: javabridge.call(self.jobject, "updateClassifier", "(Lweka/core/Instance;)V", inst.jobject) else: logger.critical(classes.get_classname(self.jobject) + " is not updateable!") def classify_instance(self, inst): """ Peforms a prediction. :param inst: the Instance to get a prediction for :type inst: Instance :return: the classification (either regression value or 0-based label index) :rtype: float """ return self.__classify(inst.jobject) def distribution_for_instance(self, inst): """ Peforms a prediction, returning the class distribution. :param inst: the Instance to get the class distribution for :type inst: Instance :return: the class distribution array :rtype: ndarray """ pred = self.__distribution(inst.jobject) return javabridge.get_env().get_double_array_elements(pred) def distributions_for_instances(self, data): """ Peforms predictions, returning the class distributions. :param data: the Instances to get the class distributions for :type data: Instances :return: the class distribution matrix, None if not a batch predictor :rtype: ndarray """ if self.is_batchpredictor: return typeconv.double_matrix_to_ndarray(self.__distributions(data.jobject)) else: return None @property def batch_size(self): """ Returns the batch size, in case this classifier is a batch predictor. :return: the batch size, None if not a batch predictor :rtype: str """ if self.is_batchpredictor: return javabridge.call(self.jobject, "getBatchSize", "()Ljava/lang/String;") else: return None @batch_size.setter def batch_size(self, size): """ Sets the batch size, in case this classifier is a batch predictor. :param size: the size of the batch :type size: str """ if self.is_batchpredictor: javabridge.call(self.jobject, "setBatchSize", "(Ljava/lang/String;)V", size) def has_efficient_batch_prediction(self): """ Returns whether the classifier implements a more efficient batch prediction. :return: True if a more efficient batch prediction is implemented, always False if not batch predictor :rtype: bool """ if self.is_batchpredictor: return javabridge.call(self.jobject, "implementsMoreEfficientBatchPrediction", "()Z") else: return False @property def graph_type(self): """ Returns the graph type if classifier implements weka.core.Drawable, otherwise -1. :return: the type :rtype: int """ if self.is_drawable: return javabridge.call(self.jobject, "graphType", "()I") else: return -1 @property def graph(self): """ Returns the graph if classifier implements weka.core.Drawable, otherwise None. :return: the generated graph string :rtype: str """ if self.is_drawable: return javabridge.call(self.jobject, "graph", "()Ljava/lang/String;") else: return None @classmethod def make_copy(cls, classifier): """ Creates a copy of the classifier. :param classifier: the classifier to copy :type classifier: Classifier :return: the copy of the classifier :rtype: Classifier """ return Classifier( jobject=javabridge.static_call( "weka/classifiers/AbstractClassifier", "makeCopy", "(Lweka/classifiers/Classifier;)Lweka/classifiers/Classifier;", classifier.jobject)) class SingleClassifierEnhancer(Classifier): """ Wrapper class for classifiers that use a single base classifier. """ def __init__(self, classname=None, jobject=None, options=None): """ Initializes the specified classifier using either the classname or the supplied JB_Object. :param classname: the classname of the classifier :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Classifier.new_instance(classname) self.enforce_type(jobject, "weka.classifiers.SingleClassifierEnhancer") super(SingleClassifierEnhancer, self).__init__(classname=classname, jobject=jobject, options=options) @property def classifier(self): """ Returns the base classifier. ;return: the base classifier :rtype: Classifier """ return Classifier(jobject=javabridge.call(self.jobject, "getClassifier", "()Lweka/classifiers/Classifier;")) @classifier.setter def classifier(self, classifier): """ Sets the base classifier. :param classifier: the base classifier to use :type classifier: Classifier """ javabridge.call(self.jobject, "setClassifier", "(Lweka/classifiers/Classifier;)V", classifier.jobject) class FilteredClassifier(SingleClassifierEnhancer): """ Wrapper class for the filtered classifier. """ def __init__(self, jobject=None, options=None): """ Initializes the specified classifier using its classname or the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ classname = "weka.classifiers.meta.FilteredClassifier" if jobject is None: jobject = Classifier.new_instance(classname) else: self.enforce_type(jobject, classname) super(FilteredClassifier, self).__init__(jobject=jobject, options=options) @property def filter(self): """ Returns the filter. :return: the filter in use :rtype: Filter """ return Filter(jobject=javabridge.call(self.jobject, "getFilter", "()Lweka/filters/Filter;")) @filter.setter def filter(self, filtr): """ Sets the filter. :param filtr: the filter to use :type filtr: Filter """ javabridge.call(self.jobject, "setFilter", "(Lweka/filters/Filter;)V", filtr.jobject) def check_for_modified_class_attribute(self, check): """ Sets whether to check for class attribute modifications. :param check: True if checking for modifications :type check: bool """ javabridge.call(self.jobject, "setDoNotCheckForModifiedClassAttribute", "(Z)V", not check) class GridSearch(SingleClassifierEnhancer): """ Wrapper class for the GridSearch meta-classifier. """ def __init__(self, jobject=None, options=None): """ Initializes the specified classifier using its classname or the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ classname = "weka.classifiers.meta.GridSearch" if jobject is None: jobject = GridSearch.new_instance(classname) if jobject is None: raise Exception( "Failed to instantiate GridSearch - package installed and jvm started with package support?") else: self.enforce_type(jobject, classname) super(GridSearch, self).__init__(jobject=jobject, options=options) self.tags_evaluation = Tags.get_tags("weka.classifiers.meta.GridSearch", "TAGS_EVALUATION") @property def evaluation(self): """ Returns the currently set statistic used for evaluation. :return: the statistic :rtype: SelectedTag """ return SelectedTag( javabridge.call(self.jobject, "getEvaluation", "()Lweka/core/SelectedTag;")) @evaluation.setter def evaluation(self, evl): """ Sets the statistic to use for evaluation. :param evl: the statistic :type evl: SelectedTag, Tag or str """ if isinstance(evl, str): evl = self.tags_evaluation.find(evl) if isinstance(evl, Tag): evl = SelectedTag(tag_id=evl.ident, tags=self.tags_evaluation) javabridge.call(self.jobject, "setEvaluation", "(Lweka/core/SelectedTag;)V", evl.jobject) @property def x(self): """ Returns a dictionary with all the current values for the X of the grid. Keys for the dictionary: property, min, max, step, base, expression Types: property=str, min=float, max=float, step=float, base=float, expression=str :return: the dictionary with the parameters :rtype: dict """ result = {} result["property"] = javabridge.call(self.jobject, "getXProperty", "()Ljava/lang/String;") result["min"] = javabridge.call(self.jobject, "getXMin", "()D") result["max"] = javabridge.call(self.jobject, "getXMax", "()D") result["step"] = javabridge.call(self.jobject, "getXStep", "()D") result["base"] = javabridge.call(self.jobject, "getXBase", "()D") result["expression"] = javabridge.call(self.jobject, "getXExpression", "()Ljava/lang/String;") return result @x.setter def x(self, d): """ Allows to configure the X of the grid with one method call. Keys for the dictionary: property, min, max, step, base, expression Types: property=str, min=float, max=float, step=float, base=float, expression=str :param d: the dictionary with the parameters :type d: dict """ if "property" in d: javabridge.call(self.jobject, "setXProperty", "(Ljava/lang/String;)V", d["property"]) if "min" in d: javabridge.call(self.jobject, "setXMin", "(D)V", d["min"]) if "max" in d: javabridge.call(self.jobject, "setXMax", "(D)V", d["max"]) if "step" in d: javabridge.call(self.jobject, "setXStep", "(D)V", d["step"]) if "base" in d: javabridge.call(self.jobject, "setXBase", "(D)V", d["base"]) if "expression" in d: javabridge.call(self.jobject, "setXExpression", "(Ljava/lang/String;)V", d["expression"]) @property def y(self): """ Returns a dictionary with all the current values for the Y of the grid. Keys for the dictionary: property, min, max, step, base, expression Types: property=str, min=float, max=float, step=float, base=float, expression=str :return: the dictionary with the parameters :rtype: dict """ result = {} result["property"] = javabridge.call(self.jobject, "getYProperty", "()Ljava/lang/String;") result["min"] = javabridge.call(self.jobject, "getYMin", "()D") result["max"] = javabridge.call(self.jobject, "getYMax", "()D") result["step"] = javabridge.call(self.jobject, "getYStep", "()D") result["base"] = javabridge.call(self.jobject, "getYBase", "()D") result["expression"] = javabridge.call(self.jobject, "getYExpression", "()Ljava/lang/String;") return result @y.setter def y(self, d): """ Allows to configure the Y of the grid with one method call. Keys for the dictionary: property, min, max, step, base, expression Types: property=str, min=float, max=float, step=float, base=float, expression=str :param d: the dictionary with the parameters :type d: dict """ if "property" in d: javabridge.call(self.jobject, "setYProperty", "(Ljava/lang/String;)V", d["property"]) if "min" in d: javabridge.call(self.jobject, "setYMin", "(D)V", d["min"]) if "max" in d: javabridge.call(self.jobject, "setYMax", "(D)V", d["max"]) if "step" in d: javabridge.call(self.jobject, "setYStep", "(D)V", d["step"]) if "base" in d: javabridge.call(self.jobject, "setYBase", "(D)V", d["base"]) if "expression" in d: javabridge.call(self.jobject, "setYExpression", "(Ljava/lang/String;)V", d["expression"]) @property def best(self): """ Returns the best classifier setup found during the th search. :return: the best classifier setup :rtype: Classifier """ return Classifier(jobject=javabridge.call(self.jobject, "getBestClassifier", "()Lweka/classifiers/Classifier;")) class MultiSearch(SingleClassifierEnhancer): """ Wrapper class for the MultiSearch meta-classifier. NB: 'multi-search-weka-package' must be installed (https://github.com/fracpete/multisearch-weka-package), version 2016.1.15 or later. """ def __init__(self, jobject=None, options=None): """ Initializes the specified classifier using its classname or the supplied JB_Object. :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ classname = "weka.classifiers.meta.MultiSearch" if jobject is None: jobject = MultiSearch.new_instance(classname) else: self.enforce_type(jobject, classname) super(MultiSearch, self).__init__(jobject=jobject, options=options) self.tags_evaluation = Tags.get_object_tags(self, "getMetricsTags") @property def evaluation(self): """ Returns the currently set statistic used for evaluation. :return: the statistic :rtype: SelectedTag """ return SelectedTag( javabridge.call(self.jobject, "getEvaluation", "()Lweka/core/SelectedTag;")) @evaluation.setter def evaluation(self, evl): """ Sets the statistic to use for evaluation. :param evl: the statistic :type evl: SelectedTag, Tag or str """ if isinstance(evl, str): evl = self.tags_evaluation.find(evl) if isinstance(evl, Tag): evl = SelectedTag(tag_id=evl.ident, tags=self.tags_evaluation) javabridge.call(self.jobject, "setEvaluation", "(Lweka/core/SelectedTag;)V", evl.jobject) @property def parameters(self): """ Returns the list of currently set search parameters. :return: the list of AbstractSearchParameter objects :rtype: list """ array = JavaArray( javabridge.call(self.jobject, "getSearchParameters", "()[Lweka/core/setupgenerator/AbstractParameter;")) result = [] for item in array: result.append(AbstractParameter(jobject=item.jobject)) return result @parameters.setter def parameters(self, params): """ Sets the list of search parameters to use. :param params: list of AbstractSearchParameter objects :type params: list """ array = JavaArray(JavaArray.new_instance("weka.core.setupgenerator.AbstractParameter", len(params))) for idx, obj in enumerate(params): array[idx] = obj.jobject javabridge.call(self.jobject, "setSearchParameters", "([Lweka/core/setupgenerator/AbstractParameter;)V", array.jobject) @property def best(self): """ Returns the best classifier setup found during the th search. :return: the best classifier setup :rtype: Classifier """ return Classifier(jobject=javabridge.call(self.jobject, "getBestClassifier", "()Lweka/classifiers/Classifier;")) class MultipleClassifiersCombiner(Classifier): """ Wrapper class for classifiers that use a multiple base classifiers. """ def __init__(self, classname=None, jobject=None, options=None): """ Initializes the specified classifier using either the classname or the supplied JB_Object. :param classname: the classname of the classifier :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: list of commandline options :type options: list """ if jobject is None: jobject = Classifier.new_instance(classname) self.enforce_type(jobject, "weka.classifiers.MultipleClassifiersCombiner") super(MultipleClassifiersCombiner, self).__init__(classname=classname, jobject=jobject, options=options) @property def classifiers(self): """ Returns the list of base classifiers. :return: the classifier list :rtype: list """ objects = javabridge.get_env().get_object_array_elements( javabridge.call(self.jobject, "getClassifiers", "()[Lweka/classifiers/Classifier;")) result = [] for obj in objects: result.append(Classifier(jobject=obj)) return result @classifiers.setter def classifiers(self, classifiers): """ Sets the base classifiers. :param classifiers: the list of base classifiers to use :type classifiers: list """ obj = [] for classifier in classifiers: obj.append(classifier.jobject) javabridge.call(self.jobject, "setClassifiers", "([Lweka/classifiers/Classifier;)V", obj) class Kernel(OptionHandler): """ Wrapper class for kernels. """ def __init__(self, classname=None, jobject=None, options=None): """ Initializes the specified kernel using either the classname or the supplied JB_Object. :param classname: the classname of the kernel :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Classifier.new_instance(classname) self.enforce_type(jobject, "weka.classifiers.functions.supportVector.Kernel") super(Kernel, self).__init__(jobject=jobject, options=options) def capabilities(self): """ Returns the capabilities of the classifier. :return: the capabilities :rtype: Capabilities """ return Capabilities(javabridge.call(self.jobject, "getCapabilities", "()Lweka/core/Capabilities;")) @property def checks_turned_off(self): """ Returns whether checks are turned off. :return: True if checks turned off :rtype: bool """ return javabridge.call(self.jobject, "getChecksTurnedOff", "()Z") @checks_turned_off.setter def checks_turned_off(self, off): """ Turns any checks on/off. :param off: True to turn off checks :type off: bool """ javabridge.call(self.jobject, "setChecksTurnedOff", "(Z)V", off) def clean(self): """ Frees the memory used by the kernel. """ javabridge.call(self.jobject, "clean", "()V") def build_kernel(self, data): """ Builds the classifier with the data. :param data: the data to train the classifier with :type data: Instances """ javabridge.call(self.jobject, "buildKernel", "(Lweka/core/Instances;)V", data.jobject) def eval(self, id1, id2, inst1): """ Computes the result of the kernel function for two instances. If id1 == -1, eval use inst1 instead of an instance in the dataset. :param id1: the index of the first instance in the dataset :type id1: int :param id2: the index of the second instance in the dataset :type id2: int :param inst1: the instance corresponding to id1 (used if id1 == -1) :type inst1: Instance """ jinst1 = None if inst1 is not None: jinst1 = inst1.jobject return javabridge.call(self.jobject, "eval", "(IILweka/core/Instance;)D", id1, id2, jinst1) @classmethod def make_copy(cls, kernel): """ Creates a copy of the kernel. :param kernel: the kernel to copy :type kernel: Kernel :return: the copy of the kernel :rtype: Kernel """ return Kernel( jobject=javabridge.static_call( "weka/classifiers/functions/supportVector/Kernel", "makeCopy", "(Lweka/classifiers/functions/supportVector/Kernel;)Lweka/classifiers/functions/supportVector/Kernel;", kernel.jobject)) class KernelClassifier(Classifier): """ Wrapper class for classifiers that have a kernel property, like SMO. """ def __init__(self, classname=None, jobject=None, options=None): """ Initializes the specified classifier using either the classname or the supplied JB_Object. :param classname: the classname of the classifier :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: list of commandline options :type options: list """ if jobject is None: jobject = Classifier.new_instance(classname) self.enforce_type(jobject, "weka.classifiers.Classifier") if not javabridge.static_call( "weka/classifiers/KernelHelper", "hasKernelProperty", "(Ljava/lang/Object;)Z", jobject): raise Exception("Does not handle a kernel: " + classes.get_classname(jobject)) super(KernelClassifier, self).__init__(classname=classname, jobject=jobject, options=options) @property def kernel(self): """ Returns the current kernel. :return: the kernel or None if none found :rtype: Kernel """ result = javabridge.static_call( "weka/classifiers/KernelHelper", "getKernel", "(Ljava/lang/Object;)Lweka/classifiers/functions/supportVector/Kernel;", self.jobject) if result is None: return None else: return Kernel(jobject=result) @kernel.setter def kernel(self, kernel): """ Sets the kernel. :param kernel: the kernel to set :type kernel: Kernel """ result = javabridge.static_call( "weka/classifiers/KernelHelper", "setKernel", "(Ljava/lang/Object;Lweka/classifiers/functions/supportVector/Kernel;)Z", self.jobject, kernel.jobject) if not result: raise Exception("Failed to set kernel!") class Prediction(JavaObject): """ Wrapper class for a prediction. """ def __init__(self, jobject): """ Initializes the wrapper. :param jobject: the prediction to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.classifiers.evaluation.Prediction") super(Prediction, self).__init__(jobject) @property def actual(self): """ Returns the actual value. :return: the actual value (internal representation) :rtype: float """ return javabridge.call(self.jobject, "actual", "()D") @property def predicted(self): """ Returns the predicted value. :return: the predicted value (internal representation) :rtype: float """ return javabridge.call(self.jobject, "predicted", "()D") @property def weight(self): """ Returns the weight. :return: the weight of the Instance that was used :rtype: float """ return javabridge.call(self.jobject, "weight", "()D") class NominalPrediction(Prediction): """ Wrapper class for a nominal prediction. """ def __init__(self, jobject): """ Initializes the wrapper. :param jobject: the prediction to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.classifiers.evaluation.NominalPrediction") super(NominalPrediction, self).__init__(jobject) @property def distribution(self): """ Returns the class distribution. :return: the class distribution list :rtype: ndarray """ return javabridge.get_env().get_double_array_elements(javabridge.call(self.jobject, "distribution", "()[D")) @property def margin(self): """ Returns the margin. :return: the margin :rtype: float """ return javabridge.call(self.jobject, "margin", "()D") class NumericPrediction(Prediction): """ Wrapper class for a numeric prediction. """ def __init__(self, jobject): """ Initializes the wrapper. :param jobject: the prediction to wrap :type jobject: JB_Object """ self.enforce_type(jobject, "weka.classifiers.evaluation.NumericPrediction") super(NumericPrediction, self).__init__(jobject) @property def error(self): """ Returns the error. :return: the error :rtype: float """ return javabridge.call(self.jobject, "error", "()D") @property def prediction_intervals(self): """ Returns the prediction intervals. :return: the intervals :rtype: ndarray """ return typeconv.double_matrix_to_ndarray(javabridge.call(self.jobject, "predictionIntervals", "()[[D")) class CostMatrix(JavaObject): """ Class for storing and manipulating a misclassification cost matrix. The element at position i,j in the matrix is the penalty for classifying an instance of class j as class i. Cost values can be fixed or computed on a per-instance basis (cost sensitive evaluation only) from the value of an attribute or an expression involving attribute(s). """ def __init__(self, matrx=None, num_classes=None): """ Initializes the matrix object. :param matrx: the matrix to copy :type matrx: CostMatrix or ndarray or JB_Object :param num_classes: the number of classes :type num_classes: int """ if matrx is not None: if isinstance(matrx, CostMatrix): jobject = javabridge.make_instance( "weka/classifiers/CostMatrix", "(Lweka/classifiers/CostMatrix;)V", matrx.jobject) super(CostMatrix, self).__init__(jobject) elif isinstance(matrx, ndarray): shp = matrx.shape if len(shp) != 2: raise Exception("Numpy array must be a 2-dimensional array!") rows, cols = shp if rows == cols: cmatrix = CostMatrix(num_classes=rows) for r in range(rows): for c in range(cols): cmatrix.set_element(r, c, matrx[r][c]) super(CostMatrix, self).__init__(cmatrix.jobject) else: raise Exception("Numpy array must be a square matrix!") elif isinstance(matrx, javabridge.JB_Object): super(CostMatrix, self).__init__(matrx) else: raise Exception( "Matrix must be either a CostMatrix or a 2-dimensional numpy array: " + str(type(matrx))) elif not num_classes is None: jobject = javabridge.make_instance( "weka/classifiers/CostMatrix", "(I)V", num_classes) super(CostMatrix, self).__init__(jobject) else: raise Exception("Either matrix or number of classes must be provided!") def apply_cost_matrix(self, data, rnd): """ Applies the cost matrix to the data. :param data: the data to apply to :type data: Instances :param rnd: the random number generator :type rnd: Random """ return Instances( javabridge.call( self.jobject, "applyCostMatrix", "(Lweka/core/Instances;Ljava/util/Random;)Lweka/core/Instances;", data.jobject, rnd.jobject)) def expected_costs(self, class_probs, inst=None): """ Calculates the expected misclassification cost for each possible class value, given class probability estimates. :param class_probs: the class probabilities :type class_probs: ndarray :return: the calculated costs :rtype: ndarray """ if inst is None: costs = javabridge.call( self.jobject, "expectedCosts", "([D)[D", javabridge.get_env().make_double_array(class_probs)) return javabridge.get_env().get_double_array_elements(costs) else: costs = javabridge.call( self.jobject, "expectedCosts", "([DLweka/core/Instance;)[D", javabridge.get_env().make_double_array(class_probs), inst.jobject) return javabridge.get_env().get_double_array_elements(costs) def get_cell(self, row, col): """ Returns the JB_Object at the specified location. :param row: the 0-based index of the row :type row: int :param col: the 0-based index of the column :type col: int :return: the object in that cell :rtype: JB_Object """ return javabridge.call( self.jobject, "getCell", "(II)Ljava/lang/Object;", row, col) def set_cell(self, row, col, obj): """ Sets the JB_Object at the specified location. Automatically unwraps JavaObject. :param row: the 0-based index of the row :type row: int :param col: the 0-based index of the column :type col: int :param obj: the object for that cell :type obj: object """ if isinstance(obj, JavaObject): obj = obj.jobject javabridge.call( self.jobject, "setCell", "(IILjava/lang/Object;)V", row, col, obj) def get_element(self, row, col, inst=None): """ Returns the value at the specified location. :param row: the 0-based index of the row :type row: int :param col: the 0-based index of the column :type col: int :param inst: the Instace :type inst: Instance :return: the value in that cell :rtype: float """ if inst is None: return javabridge.call( self.jobject, "getElement", "(II)D", row, col) else: return javabridge.call( self.jobject, "getElement", "(IILweka/core/Instance;)D", row, col, inst.jobject) def set_element(self, row, col, value): """ Sets the float value at the specified location. :param row: the 0-based index of the row :type row: int :param col: the 0-based index of the column :type col: int :param value: the float value for that cell :type value: float """ javabridge.call( self.jobject, "setElement", "(IID)V", row, col, value) def get_max_cost(self, class_value, inst=None): """ Gets the maximum cost for a particular class value. :param class_value: the class value to get the maximum cost for :type class_value: int :param inst: the Instance :type inst: Instance :return: the cost :rtype: float """ if inst is None: return javabridge.call( self.jobject, "getMaxCost", "(I)D", class_value) else: return javabridge.call( self.jobject, "getElement", "(ILweka/core/Instance;)D", class_value, inst.jobject) def initialize(self): """ Initializes the matrix. """ javabridge.call(self.jobject, "initialize", "()V") def normalize(self): """ Normalizes the matrix. """ javabridge.call(self.jobject, "normalize", "()V") @property def num_columns(self): """ Returns the number of columns. :return: the number of columns :rtype: int """ return javabridge.call(self.jobject, "numColumns", "()I") @property def num_rows(self): """ Returns the number of rows. :return: the number of rows :rtype: int """ return javabridge.call(self.jobject, "numRows", "()I") @property def size(self): """ Returns the number of rows/columns. :return: the number of rows/columns :rtype: int """ return javabridge.call(self.jobject, "size", "()I") def to_matlab(self): """ Returns the matrix in Matlab format. :return: the matrix as Matlab formatted string :rtype: str """ return javabridge.call(self.jobject, "toMatlab", "()Ljava/lang/String;") @classmethod def parse_matlab(cls, matlab): """ Parses the costmatrix definition in matlab format and returns a matrix. :param matlab: the matlab matrix string, eg [1 2; 3 4]. :type matlab: str :return: the generated matrix :rtype: CostMatrix """ return CostMatrix( matrx=javabridge.static_call( "weka/classifiers/CostMatrix", "parseMatlab", "(Ljava/lang/String;)Lweka/classifiers/CostMatrix;", matlab)) class Evaluation(JavaObject): """ Evaluation class for classifiers. """ def __init__(self, data, cost_matrix=None): """ Initializes an Evaluation object. :param data: the data to use to initialize the priors with :type data: Instances :param cost_matrix: the cost matrix to use for initializing :type cost_matrix: CostMatrix """ if cost_matrix is None: jobject = javabridge.make_instance( "weka/classifiers/EvaluationWrapper", "(Lweka/core/Instances;)V", data.jobject) else: jobject = javabridge.make_instance( "weka/classifiers/EvaluationWrapper", "(Lweka/core/Instances;Lweka/classifiers/CostMatrix;)V", data.jobject, cost_matrix.jobject) self.wrapper = jobject jobject = javabridge.call(jobject, "getEvaluation", "()Lweka/classifiers/Evaluation;") super(Evaluation, self).__init__(jobject) def crossvalidate_model(self, classifier, data, num_folds, rnd, output=None): """ Crossvalidates the model using the specified data, number of folds and random number generator wrapper. :param classifier: the classifier to cross-validate :type classifier: Classifier :param data: the data to evaluate on :type data: Instances :param num_folds: the number of folds :type num_folds: int :param rnd: the random number generator to use :type rnd: Random :param output: the output generator to use :type output: PredictionOutput """ if output is None: generator = [] else: generator = [output.jobject] javabridge.call( self.jobject, "crossValidateModel", "(Lweka/classifiers/Classifier;Lweka/core/Instances;ILjava/util/Random;[Ljava/lang/Object;)V", classifier.jobject, data.jobject, num_folds, rnd.jobject, generator) def evaluate_train_test_split(self, classifier, data, percentage, rnd=None, output=None): """ Splits the data into train and test, builds the classifier with the training data and evaluates it against the test set. :param classifier: the classifier to cross-validate :type classifier: Classifier :param data: the data to evaluate on :type data: Instances :param percentage: the percentage split to use (amount to use for training) :type percentage: double :param rnd: the random number generator to use, if None the order gets preserved :type rnd: Random :param output: the output generator to use :type output: PredictionOutput """ train_inst, test_inst = data.train_test_split(percentage, rnd=rnd) cls = Classifier.make_copy(classifier) cls.build_classifier(train_inst) self.test_model(cls, test_inst, output=output) def test_model(self, classifier, data, output=None): """ Evaluates the built model using the specified test data and returns the classifications. :param classifier: the trained classifier to evaluate :type classifier: Classifier :param data: the data to evaluate on :type data: Instances :param output: the output generator to use :type output: PredictionOutput :return: the classifications :rtype: ndarray """ if output is None: generator = [] else: output.header = data generator = [output.jobject] cls = javabridge.call( self.jobject, "evaluateModel", "(Lweka/classifiers/Classifier;Lweka/core/Instances;[Ljava/lang/Object;)[D", classifier.jobject, data.jobject, generator) if cls is None: return None else: return javabridge.get_env().get_double_array_elements(cls) def test_model_once(self, classifier, inst): """ Evaluates the built model using the specified test instance and returns the classification. :param classifier: the classifier to cross-validate :type classifier: Classifier :param inst: the Instance to evaluate on :type inst: Instances :return: the classification :rtype: float """ return javabridge.call( self.jobject, "evaluateModelOnce", "(Lweka/classifiers/Classifier;Lweka/core/Instance;)D", classifier.jobject, inst.jobject) def summary(self, title=None, complexity=False): """ Generates a summary. :param title: optional title :type title: str :param complexity: whether to print the complexity information as well :type complexity: bool :return: the summary :rtype: str """ if title is None: return javabridge.call( self.jobject, "toSummaryString", "()Ljava/lang/String;") else: return javabridge.call( self.jobject, "toSummaryString", "(Ljava/lang/String;Z)Ljava/lang/String;", title, complexity) def class_details(self, title=None): """ Generates the class details. :param title: optional title :type title: str :return: the details :rtype: str """ if title is None: return javabridge.call( self.jobject, "toClassDetailsString", "()Ljava/lang/String;") else: return javabridge.call( self.jobject, "toClassDetailsString", "(Ljava/lang/String;)Ljava/lang/String;", title) def matrix(self, title=None): """ Generates the confusion matrix. :param title: optional title :type title: str :return: the matrix :rtype: str """ if title is None: return javabridge.call(self.jobject, "toMatrixString", "()Ljava/lang/String;") else: return javabridge.call(self.jobject, "toMatrixString", "(Ljava/lang/String;)Ljava/lang/String;", title) def area_under_prc(self, class_index): """ Returns the area under precision recall curve. :param class_index: the 0-based index of the class label :type class_index: int :return: the area :rtype: float """ return javabridge.call(self.jobject, "areaUnderPRC", "(I)D", class_index) @property def weighted_area_under_prc(self): """ Returns the weighted area under precision recall curve. :return: the weighted area :rtype: float """ return javabridge.call(self.jobject, "weightedAreaUnderPRC", "()D") def area_under_roc(self, class_index): """ Returns the area under receiver operators characteristics curve. :param class_index: the 0-based index of the class label :type class_index: int :return: the area :rtype: float """ return javabridge.call(self.jobject, "areaUnderROC", "(I)D", class_index) @property def weighted_area_under_roc(self): """ Returns the weighted area under receiver operator characteristic curve. :return: the weighted area :rtype: float """ return javabridge.call(self.jobject, "weightedAreaUnderROC", "()D") @property def avg_cost(self): """ Returns the average cost. :return: the cost :rtype: float """ return javabridge.call(self.jobject, "avgCost", "()D") @property def total_cost(self): """ Returns the total cost. :return: the cost :rtype: float """ return javabridge.call(self.jobject, "totalCost", "()D") @property def confusion_matrix(self): """ Returns the confusion matrix. :return: the matrix :rtype: ndarray """ return typeconv.double_matrix_to_ndarray(javabridge.call(self.jobject, "confusionMatrix", "()[[D")) @property def correct(self): """ Returns the correct count (nominal classes). :return: the count :rtype: float """ return javabridge.call(self.jobject, "correct", "()D") @property def incorrect(self): """ Returns the incorrect count (nominal classes). :return: the count :rtype: float """ return javabridge.call(self.jobject, "incorrect", "()D") @property def unclassified(self): """ Returns the unclassified count. :return: the count :rtype: float """ return javabridge.call(self.jobject, "unclassified", "()D") @property def num_instances(self): """ Returns the number of instances that had a known class value. :return: the number of instances :rtype: float """ return javabridge.call(self.jobject, "numInstances", "()D") @property def percent_correct(self): """ Returns the percent correct (nominal classes). :return: the percentage :rtype: float """ return javabridge.call(self.jobject, "pctCorrect", "()D") @property def percent_incorrect(self): """ Returns the percent incorrect (nominal classes). :return: the percentage :rtype: float """ return javabridge.call(self.jobject, "pctIncorrect", "()D") @property def percent_unclassified(self): """ Returns the percent unclassified. :return: the percentage :rtype: float """ return javabridge.call(self.jobject, "pctUnclassified", "()D") @property def correlation_coefficient(self): """ Returns the correlation coefficient (numeric classes). :return: the coefficient :rtype: float """ return javabridge.call(self.jobject, "correlationCoefficient", "()D") def matthews_correlation_coefficient(self, class_index): """ Returns the Matthews correlation coefficient (nominal classes). :param class_index: the 0-based index of the class label :type class_index: int :return: the coefficient :rtype: float """ return javabridge.call(self.jobject, "matthewsCorrelationCoefficient", "(I)D", class_index) @property def weighted_matthews_correlation(self): """ Returns the weighted Matthews correlation (nominal classes). :return: the correlation :rtype: float """ return javabridge.call(self.jobject, "weightedMatthewsCorrelation", "()D") @property def coverage_of_test_cases_by_predicted_regions(self): """ Returns the coverage of the test cases by the predicted regions at the confidence level specified when evaluation was performed. :return: the coverage :rtype: float """ return javabridge.call(self.jobject, "coverageOfTestCasesByPredictedRegions", "()D") @property def size_of_predicted_regions(self): """ Returns the average size of the predicted regions, relative to the range of the target in the training data, at the confidence level specified when evaluation was performed. :return:the size of the regions :rtype: float """ return javabridge.call(self.jobject, "sizeOfPredictedRegions", "()D") @property def error_rate(self): """ Returns the error rate (numeric classes). :return: the rate :rtype: float """ return javabridge.call(self.jobject, "errorRate", "()D") @property def mean_absolute_error(self): """ Returns the mean absolute error. :return: the error :rtype: float """ return javabridge.call(self.jobject, "meanAbsoluteError", "()D") @property def relative_absolute_error(self): """ Returns the relative absolute error. :return: the error :rtype: float """ return javabridge.call(self.jobject, "relativeAbsoluteError", "()D") @property def root_mean_squared_error(self): """ Returns the root mean squared error. :return: the error :rtype: float """ return javabridge.call(self.jobject, "rootMeanSquaredError", "()D") @property def root_relative_squared_error(self): """ Returns the root relative squared error. :return: the error :rtype: float """ return javabridge.call(self.jobject, "rootRelativeSquaredError", "()D") @property def root_mean_prior_squared_error(self): """ Returns the root mean prior squared error. :return: the error :rtype: float """ return javabridge.call(self.jobject, "rootMeanPriorSquaredError", "()D") @property def mean_prior_absolute_error(self): """ Returns the mean prior absolute error. :return: the error :rtype: float """ return javabridge.call(self.jobject, "meanPriorAbsoluteError", "()D") def false_negative_rate(self, class_index): """ Returns the false negative rate. :param class_index: the 0-based index of the class label :type class_index: int :return: the rate :rtype: float """ return javabridge.call(self.jobject, "falseNegativeRate", "(I)D", class_index) @property def weighted_false_negative_rate(self): """ Returns the weighted false negative rate. :return: the rate :rtype: float """ return javabridge.call(self.jobject, "weightedFalseNegativeRate", "()D") def false_positive_rate(self, class_index): """ Returns the false positive rate. :param class_index: the 0-based index of the class label :type class_index: int :return: the rate :rtype: float """ return javabridge.call(self.jobject, "falsePositiveRate", "(I)D", class_index) @property def weighted_false_positive_rate(self): """ Returns the weighted false positive rate. :return: the rate :rtype: float """ return javabridge.call(self.jobject, "weightedFalsePositiveRate", "()D") def num_false_negatives(self, class_index): """ Returns the number of false negatives. :param class_index: the 0-based index of the class label :type class_index: int :return: the count :rtype: float """ return javabridge.call(self.jobject, "numFalseNegatives", "(I)D", class_index) def true_negative_rate(self, class_index): """ Returns the true negative rate. :param class_index: the 0-based index of the class label :type class_index: int :return: the rate :rtype: float """ return javabridge.call(self.jobject, "trueNegativeRate", "(I)D", class_index) @property def weighted_true_negative_rate(self): """ Returns the weighted true negative rate. :return: the rate :rtype: float """ return javabridge.call(self.jobject, "weightedTrueNegativeRate", "()D") def num_true_negatives(self, class_index): """ Returns the number of true negatives. :param class_index: the 0-based index of the class label :type class_index: int :return: the count :rtype: float """ return javabridge.call(self.jobject, "numTrueNegatives", "(I)D", class_index) def num_false_positives(self, class_index): """ Returns the number of false positives. :param class_index: the 0-based index of the class label :type class_index: int :return: the count :rtype: float """ return javabridge.call(self.jobject, "numFalsePositives", "(I)D", class_index) def true_positive_rate(self, class_index): """ Returns the true positive rate. :param class_index: the 0-based index of the class label :type class_index: int :return: the rate :rtype: float """ return javabridge.call(self.jobject, "truePositiveRate", "(I)D", class_index) @property def weighted_true_positive_rate(self): """ Returns the weighted true positive rate. :return: the rate :rtype: float """ return javabridge.call(self.jobject, "weightedTruePositiveRate", "()D") def num_true_positives(self, class_index): """ Returns the number of true positives. :param class_index: the 0-based index of the class label :type class_index: int :return: the count :rtype: float """ return javabridge.call(self.jobject, "numTruePositives", "(I)D", class_index) def f_measure(self, class_index): """ Returns the f measure. :param class_index: the 0-based index of the class label :type class_index: int :return: the measure :rtype: float """ return javabridge.call(self.jobject, "fMeasure", "(I)D", class_index) @property def weighted_f_measure(self): """ Returns the weighted f measure. :return: the measure :rtype: float """ return javabridge.call(self.jobject, "weightedFMeasure", "()D") @property def unweighted_macro_f_measure(self): """ Returns the unweighted macro-averaged F-measure. :return: the measure :rtype: float """ return javabridge.call(self.jobject, "unweightedMacroFmeasure", "()D") @property def unweighted_micro_f_measure(self): """ Returns the unweighted micro-averaged F-measure. :return: the measure :rtype: float """ return javabridge.call(self.jobject, "unweightedMicroFmeasure", "()D") def precision(self, class_index): """ Returns the precision. :param class_index: the 0-based index of the class label :type class_index: int :return: the precision :rtype: float """ return javabridge.call(self.jobject, "precision", "(I)D", class_index) @property def weighted_precision(self): """ Returns the weighted precision. :return: the precision :rtype: float """ return javabridge.call(self.jobject, "weightedPrecision", "()D") def recall(self, class_index): """ Returns the recall. :param class_index: the 0-based index of the class label :type class_index: int :return: the recall :rtype: float """ return javabridge.call(self.jobject, "recall", "(I)D", class_index) @property def weighted_recall(self): """ Returns the weighted recall. :return: the recall :rtype: float """ return javabridge.call(self.jobject, "weightedRecall", "()D") @property def kappa(self): """ Returns kappa. :return: kappa :rtype: float """ return javabridge.call(self.jobject, "kappa", "()D") @property def kb_information(self): """ Returns KB information. :return: the information :rtype: float """ return javabridge.call(self.jobject, "KBInformation", "()D") @property def kb_mean_information(self): """ Returns KB mean information. :return: the information :rtype: float """ return javabridge.call(self.jobject, "KBMeanInformation", "()D") @property def kb_relative_information(self): """ Returns KB relative information. :return: the information :rtype: float """ return javabridge.call(self.jobject, "KBRelativeInformation", "()D") @property def sf_entropy_gain(self): """ Returns the total SF, which is the null model entropy minus the scheme entropy. :return: the gain :rtype: float """ return javabridge.call(self.jobject, "SFEntropyGain", "()D") @property def sf_mean_entropy_gain(self): """ Returns the SF per instance, which is the null model entropy minus the scheme entropy, per instance. :return: the gain :rtype: float """ return javabridge.call(self.jobject, "SFMeanEntropyGain", "()D") @property def sf_mean_prior_entropy(self): """ Returns the entropy per instance for the null model. :return: the entropy :rtype: float """ return javabridge.call(self.jobject, "SFMeanPriorEntropy", "()D") @property def sf_mean_scheme_entropy(self): """ Returns the entropy per instance for the scheme. :return: the entropy :rtype: float """ return javabridge.call(self.jobject, "SFMeanSchemeEntropy", "()D") @property def class_priors(self): """ Returns the class priors. :return: the priors :rtype: ndarray """ return javabridge.get_env().get_double_array_elements(javabridge.call(self.jobject, "getClassPriors", "()[D")) @class_priors.setter def class_priors(self, data): """ Sets the class priors derived from the dataset. :param data: the dataset to derive the priors from :type data: Instances """ javabridge.call(self.jobject, "setClassPriors", "(Lweka/core/Instances;)V", data) @property def header(self): """ Returns the header format. :return: the header format :rtype: Instances """ return Instances(javabridge.call(self.jobject, "getHeader", "()Lweka/core/Instances;")) @property def discard_predictions(self): """ Returns whether to discard predictions (saves memory). :return: True if to discard :rtype: bool """ return javabridge.call(self.jobject, "getDiscardPredictions", "()Z") @discard_predictions.setter def discard_predictions(self, discard): """ Sets whether to discard predictions (saves memory). :param discard: True if to discard predictions :type discard: bool """ javabridge.call(self.jobject, "setDiscardPredictions", "(Z)V", discard) @property def predictions(self): """ Returns the predictions. :return: the predictions. None if not available :rtype: list """ preds = javabridge.get_collection_wrapper( javabridge.call(self.jobject, "predictions", "()Ljava/util/ArrayList;")) if self.discard_predictions: result = None else: result = [] for pred in preds: if javabridge.is_instance_of(pred, "weka/classifiers/evaluation/NominalPrediction"): result.append(NominalPrediction(pred)) elif javabridge.is_instance_of(pred, "weka/classifiers/evaluation/NumericPrediction"): result.append(NumericPrediction(pred)) else: result.append(Prediction(pred)) return result @classmethod def evaluate_model(cls, classifier, args): """ Evaluates the classifier with the given options. :param classifier: the classifier instance to use :type classifier: Classifier :param args: the command-line arguments to use :type args: list :return: the evaluation string :rtype: str """ return javabridge.static_call( "Lweka/classifiers/Evaluation;", "evaluateModel", "(Lweka/classifiers/Classifier;[Ljava/lang/String;)Ljava/lang/String;", classifier.jobject, args) class PredictionOutput(OptionHandler): """ For collecting predictions and generating output from. Must be derived from weka.classifiers.evaluation.output.prediction.AbstractOutput """ def __init__(self, classname="weka.classifiers.evaluation.output.prediction.PlainText", jobject=None, options=None): """ Initializes the specified output generator using either the classname or the supplied JB_Object. :param classname: the classname of the generator :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = PredictionOutput.new_instance(classname) self.enforce_type(jobject, "weka.classifiers.evaluation.output.prediction.AbstractOutput") super(PredictionOutput, self).__init__(jobject=jobject, options=options) buf = javabridge.make_instance("java/lang/StringBuffer", "()V") javabridge.call(self.jobject, "setBuffer", "(Ljava/lang/StringBuffer;)V", buf) @property def header(self): """ Returns the header format. :return: The dataset format :rtype: Instances """ return Instances(javabridge.call(self.jobject, "getHeader", "()Lweka/core/Instances;")) @header.setter def header(self, data): """ Sets the header format. :param data: The dataset format :type data: Instances """ javabridge.call(self.jobject, "setHeader", "(Lweka/core/Instances;)V", data.jobject) def print_header(self): """ Prints the header to the buffer. """ javabridge.call(self.jobject, "printHeader", "()V") def print_footer(self): """ Prints the footer to the buffer. """ javabridge.call(self.jobject, "printFooter", "()V") def print_all(self, cls, data): """ Prints the header, classifications and footer to the buffer. :param cls: the classifier :type cls: Classifier :param data: the test data :type data: Instances """ javabridge.call( self.jobject, "print", "(Lweka/classifiers/Classifier;Lweka/core/Instances;)V", cls.jobject, data.jobject) def print_classifications(self, cls, data): """ Prints the classifications to the buffer. :param cls: the classifier :type cls: Classifier :param data: the test data :type data: Instances """ javabridge.call( self.jobject, "printClassifications", "(Lweka/classifiers/Classifier;Lweka/core/Instances;)V", cls.jobject, data.jobject) def print_classification(self, cls, inst, index): """ Prints the classification to the buffer. :param cls: the classifier :type cls: Classifier :param inst: the test instance :type inst: Instance :param index: the 0-based index of the test instance :type index: int """ javabridge.call( self.jobject, "printClassification", "(Lweka/classifiers/Classifier;Lweka/core/Instance;I)V", cls.jobject, inst.jobject, index) def buffer_content(self): """ Returns the content of the buffer as string. :return: The buffer content :rtype: str """ return javabridge.to_string(javabridge.call(self.jobject, "getBuffer", "()Ljava/lang/StringBuffer;")) def __str__(self): """ Returns the content of the buffer. :return: the current buffer content :rtype: str """ return self.buffer_content() def predictions_to_instances(data, preds): """ Turns the predictions turned into an Instances object. :param data: the original dataset format :type data: Instances :param preds: the predictions to convert :type preds: list :return: the predictions, None if no predictions present :rtype: Instances """ if len(preds) == 0: return None is_numeric = isinstance(preds[0], NumericPrediction) # create header atts = [] if is_numeric: atts.append(Attribute.create_numeric("index")) atts.append(Attribute.create_numeric("weight")) atts.append(Attribute.create_numeric("actual")) atts.append(Attribute.create_numeric("predicted")) atts.append(Attribute.create_numeric("error")) else: atts.append(Attribute.create_numeric("index")) atts.append(Attribute.create_numeric("weight")) atts.append(data.class_attribute.copy(name="actual")) atts.append(data.class_attribute.copy(name="predicted")) atts.append(Attribute.create_nominal("error", ["no", "yes"])) atts.append(Attribute.create_numeric("classification")) for i in range(data.class_attribute.num_values): atts.append(Attribute.create_numeric("distribution-" + data.class_attribute.value(i))) result = Instances.create_instances("Predictions", atts, len(preds)) count = 0 for pred in preds: count += 1 if is_numeric: values = array([count, pred.weight, pred.actual, pred.predicted, pred.error]) else: if pred.actual == pred.predicted: error = 0.0 else: error = 1.0 l = [count, pred.weight, pred.actual, pred.predicted, error, max(pred.distribution)] for i in range(data.class_attribute.num_values): l.append(pred.distribution[i]) values = array(l) inst = Instance.create_instance(values) result.add_instance(inst) return result def main(): """ Runs a classifier from the command-line. Calls JVM start/stop automatically. Use -h to see all options. """ parser = argparse.ArgumentParser( description='Performs classification/regression from the command-line. Calls JVM start/stop automatically.') parser.add_argument("-j", metavar="classpath", dest="classpath", help="additional classpath, jars/directories") parser.add_argument("-X", metavar="heap", dest="heap", help="max heap size for jvm, e.g., 512m") parser.add_argument("-t", metavar="train", dest="train", required=True, help="Training set file") parser.add_argument("-T", metavar="test", dest="test", help="Test set file") parser.add_argument("-c", metavar="class index", dest="classindex", help="1-based class attribute index") parser.add_argument("-d", metavar="outmodel", dest="outmodel", help="model output file name") parser.add_argument("-l", metavar="inmodel", dest="inmodel", help="model input file name") parser.add_argument("-x", metavar="num folds", dest="numfolds", help="number of folds for cross-validation") parser.add_argument("-s", metavar="seed", dest="seed", help="seed value for randomization") parser.add_argument("-v", action="store_true", dest="notrainstats", help="no statistics for training") parser.add_argument("-o", action="store_true", dest="onlystats", help="only statistics, don't output model") parser.add_argument("-i", action="store_true", dest="irstats", help="output information retrieval statistics") parser.add_argument("-k", action="store_true", dest="itstats", help="output information theoretic statistics") parser.add_argument("-m", metavar="costmatrix", dest="costmatrix", help="cost matrix file") parser.add_argument("-g", metavar="graph", dest="graph", help="output file for graph (if supported)") parser.add_argument("classifier", help="classifier classname, e.g., weka.classifiers.trees.J48") parser.add_argument("option", nargs=argparse.REMAINDER, help="additional classifier options") parsed = parser.parse_args() jars = [] if parsed.classpath is not None: jars = parsed.classpath.split(os.pathsep) params = [] if parsed.train is not None: params.extend(["-t", parsed.train]) if parsed.test is not None: params.extend(["-T", parsed.test]) if parsed.classindex is not None: params.extend(["-c", parsed.classindex]) if parsed.outmodel is not None: params.extend(["-d", parsed.outmodel]) if parsed.inmodel is not None: params.extend(["-l", parsed.inmodel]) if parsed.numfolds is not None: params.extend(["-x", parsed.numfolds]) if parsed.seed is not None: params.extend(["-s", parsed.seed]) if parsed.notrainstats: params.append("-v") if parsed.onlystats: params.append("-o") if parsed.irstats: params.append("-i") if parsed.itstats: params.append("-k") if parsed.costmatrix is not None: params.extend(["-m", parsed.costmatrix]) if parsed.graph is not None: params.extend(["-g", parsed.graph]) jvm.start(jars, max_heap_size=parsed.heap, packages=True) logger.debug("Commandline: " + join_options(sys.argv[1:])) try: classifier = Classifier(classname=parsed.classifier) if len(parsed.option) > 0: classifier.options = parsed.option print(Evaluation.evaluate_model(classifier, params)) except Exception as e: print(e) finally: jvm.stop() if __name__ == "__main__": try: main() except Exception as ex: print(ex) python-weka-wrapper3-0.1.3/python/weka/datagenerators.py0000644017267001726700000001535312712225046025007 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # datagenerators.py # Copyright (C) 2014-201 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import os import sys import argparse import weka.core.jvm as jvm import weka.core.classes as classes from weka.core.classes import OptionHandler, join_options, to_commandline, from_commandline from weka.core.dataset import Instances, Instance # logging setup logger = logging.getLogger("weka.datagenerators") class DataGenerator(OptionHandler): """ Wrapper class for datagenerators. """ def __init__(self, classname="weka.datagenerators.classifiers.classification.Agrawal", jobject=None, options=None): """ Initializes the specified datagenerator using either the classname or the supplied JB_Object. :param classname: the classname of the datagenerator :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object """ if jobject is None: jobject = DataGenerator.new_instance(classname) self.enforce_type(jobject, "weka.datagenerators.DataGenerator") super(DataGenerator, self).__init__(jobject=jobject, options=options) def define_data_format(self): """ Returns the data format. :return: the data format :rtype: Instances """ data = javabridge.call(self.jobject, "defineDataFormat", "()Lweka/core/Instances;") if data is None: return None else: return Instances(data) @property def single_mode_flag(self): """ Returns whether data is generated row by row (True) or in one go (False). :return: whether incremental :rtype: bool """ return javabridge.call(self.jobject, "getSingleModeFlag", "()Z") @property def dataset_format(self): """ Returns the dataset format. :return: the format :rtype: Instances """ data = javabridge.call(self.jobject, "getDatasetFormat", "()Lweka/core/Instances;") if data is None: return None else: return Instances(data) @dataset_format.setter def dataset_format(self, inst): """ Sets the dataset format. :param inst: the Instances to use as dataset format :type inst: Instances """ javabridge.call(self.jobject, "setDatasetFormat", "(Lweka/core/Instances;)V", inst.jobject) def generate_start(self): """ Returns a "start" string. :return: the start comment :rtype: str """ return javabridge.call(self.jobject, "generateStart", "()Ljava/lang/String;") @property def num_examples_act(self): """ Returns a actual number of examples to generate. :return: the number of examples :rtype: int """ return javabridge.call(self.jobject, "getNumExamplesAct", "()I") def generate_example(self): """ Returns a single Instance. :return: the next example :rtype: Instance """ data = javabridge.call(self.jobject, "generateExample", "()Lweka/core/Instance;") if data is None: return None else: return Instance(data) def generate_examples(self): """ Returns complete dataset. :return: the generated dataset :rtype: Instances """ data = javabridge.call(self.jobject, "generateExamples", "()Lweka/core/Instances;") if data is None: return None else: return Instances(data) def generate_finish(self): """ Returns a "finish" string. :return: a finish comment :rtype: str """ return javabridge.call(self.jobject, "generateFinish", "()Ljava/lang/String;") @classmethod def make_data(cls, generator, args): """ Generates data using the generator and commandline arguments. :param generator: the generator instance to use :type generator: DataGenerator :param args: the command-line arguments :type args: list """ javabridge.static_call( "Lweka/datagenerators/DataGenerator;", "makeData", "(Lweka/datagenerators/DataGenerator;[Ljava/lang/String;)V", generator.jobject, args) @classmethod def make_copy(cls, generator): """ Creates a copy of the generator. :param generator: the generator to copy :type generator: DataGenerator :return: the copy of the generator :rtype: DataGenerator """ return from_commandline( to_commandline(generator), classname=classes.get_classname(DataGenerator())) def main(): """ Runs a datagenerator from the command-line. Calls JVM start/stop automatically. Use -h to see all options. """ parser = argparse.ArgumentParser( description='Executes a data generator from the command-line. Calls JVM start/stop automatically.') parser.add_argument("-j", metavar="classpath", dest="classpath", help="additional classpath, jars/directories") parser.add_argument("-X", metavar="heap", dest="heap", help="max heap size for jvm, e.g., 512m") parser.add_argument("datagenerator", help="data generator classname, e.g., " + "weka.datagenerators.classifiers.classification.LED24") parser.add_argument("option", nargs=argparse.REMAINDER, help="additional data generator options") parsed = parser.parse_args() jars = [] if parsed.classpath is not None: jars = parsed.classpath.split(os.pathsep) jvm.start(jars, max_heap_size=parsed.heap, packages=True) logger.debug("Commandline: " + join_options(sys.argv[1:])) try: generator = DataGenerator(classname=parsed.datagenerator) if len(parsed.option) > 0: generator.options = parsed.option DataGenerator.make_data(generator, parsed.option) except Exception as e: print(e) finally: jvm.stop() if __name__ == "__main__": try: main() except Exception as ex: print(ex) python-weka-wrapper3-0.1.3/python/weka/experiments.py0000644017267001726700000007463212713764575024374 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # experiments.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import logging import javabridge from weka.core.classes import OptionHandler, Range, from_commandline from weka.core.dataset import Instances from weka.classifiers import Classifier # logging setup logger = logging.getLogger("weka.experiments") class Experiment(OptionHandler): """ Wrapper class for an experiment. """ def __init__(self, classname="weka.experiment.Experiment", jobject=None, options=None): """ Initializes the specified experiment using either the classname or the supplied JB_Object. :param classname: the classname of the experiment :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to use :type options: list """ if jobject is None: jobject = Experiment.new_instance(classname) super(Experiment, self).__init__(jobject=jobject, options=options) class SimpleExperiment(OptionHandler): """ Ancestor for simple experiments. See following URL for how to use the Experiment API: http://weka.wikispaces.com/Using+the+Experiment+API """ def __init__(self, datasets, classifiers, jobject=None, classification=True, runs=10, result=None): """ Initializes the experiment. :param datasets: the filenames of datasets to use in the experiment :type datasets: list :param classifiers: the Classifier objects or commandline strings to use in the experiment :type classifiers: list :param jobject: the Java Object to use :type jobject: JB_Object :param classification: whether to perform classification or regression :type classification:bool :param runs: the number of runs to perform :type runs: int :param result: the filename of the file to store the results in :type result: str """ if not jobject is None: self.enforce_type(jobject, "weka.experiment.Experiment") if jobject is None: jobject = javabridge.make_instance("weka/experiment/Experiment", "()V") self.classification = classification self.runs = runs self.datasets = datasets[:] self.classifiers = classifiers[:] self.result = result super(SimpleExperiment, self).__init__(jobject=jobject) def configure_splitevaluator(self): """ Configures and returns the SplitEvaluator and Classifier instance as tuple. :return: evaluator and classifier :rtype: tuple """ if self.classification: speval = javabridge.make_instance("weka/experiment/ClassifierSplitEvaluator", "()V") else: speval = javabridge.make_instance("weka/experiment/RegressionSplitEvaluator", "()V") classifier = javabridge.call(speval, "getClassifier", "()Lweka/classifiers/Classifier;") return speval, classifier def configure_resultproducer(self): """ Configures and returns the ResultProducer and PropertyPath as tuple. :return: producer and property path :rtype: tuple """ raise Exception("Not implemented!") def setup(self): """ Initializes the experiment. """ # basic options javabridge.call( self.jobject, "setPropertyArray", "(Ljava/lang/Object;)V", javabridge.get_env().make_object_array(0, javabridge.get_env().find_class("weka/classifiers/Classifier"))) javabridge.call( self.jobject, "setUsePropertyIterator", "(Z)V", True) javabridge.call( self.jobject, "setRunLower", "(I)V", 1) javabridge.call( self.jobject, "setRunUpper", "(I)V", self.runs) # setup result producer rproducer, prop_path = self.configure_resultproducer() javabridge.call( self.jobject, "setResultProducer", "(Lweka/experiment/ResultProducer;)V", rproducer) javabridge.call( self.jobject, "setPropertyPath", "([Lweka/experiment/PropertyNode;)V", prop_path) # classifiers classifiers = javabridge.get_env().make_object_array( len(self.classifiers), javabridge.get_env().find_class("weka/classifiers/Classifier")) for i, classifier in enumerate(self.classifiers): if type(classifier) is Classifier: javabridge.get_env().set_object_array_element( classifiers, i, classifier.jobject) else: javabridge.get_env().set_object_array_element( classifiers, i, from_commandline(classifier).jobject) javabridge.call( self.jobject, "setPropertyArray", "(Ljava/lang/Object;)V", classifiers) # datasets datasets = javabridge.make_instance("javax/swing/DefaultListModel", "()V") for dataset in self.datasets: f = javabridge.make_instance("java/io/File", "(Ljava/lang/String;)V", dataset) javabridge.call(datasets, "addElement", "(Ljava/lang/Object;)V", f) javabridge.call( self.jobject, "setDatasets", "(Ljavax/swing/DefaultListModel;)V", datasets) # output file if str(self.result).lower().endswith(".arff"): rlistener = javabridge.make_instance("weka/experiment/InstancesResultListener", "()V") elif str(self.result).lower().endswith(".csv"): rlistener = javabridge.make_instance("weka/experiment/CSVResultListener", "()V") else: raise Exception("Unhandled output format for results: " + self.result) rfile = javabridge.make_instance("java/io/File", "(Ljava/lang/String;)V", self.result) javabridge.call( rlistener, "setOutputFile", "(Ljava/io/File;)V", rfile) javabridge.call( self.jobject, "setResultListener", "(Lweka/experiment/ResultListener;)V", rlistener) def run(self): """ Executes the experiment. """ logger.info("Initializing...") javabridge.call(self.jobject, "initialize", "()V") logger.info("Running...") javabridge.call(self.jobject, "runExperiment", "()V") logger.info("Finished...") javabridge.call(self.jobject, "postProcess", "()V") def experiment(self): """ Returns the internal experiment, if set up, otherwise None. :return: the internal experiment :rtype: Experiment """ if self.jobject is None: return None else: return Experiment(jobject=self.jobject) @classmethod def load(cls, filename): """ Loads the experiment from disk. :param filename: the filename of the experiment to load :type filename: str :return: the experiment :rtype: Experiment """ jobject = javabridge.static_call( "weka/experiment/Experiment", "read", "(Ljava/lang/String;)Lweka/experiment/Experiment;", filename) return Experiment(jobject=jobject) @classmethod def save(cls, filename, experiment): """ Saves the experiment to disk. :param filename: the filename to save the experiment to :type filename: str :param experiment: the Experiment to save :type experiment: Experiment """ javabridge.static_call( "weka/experiment/Experiment", "write", "(Ljava/lang/String;Lweka/experiment/Experiment;)V", filename, experiment.jobject) class SimpleCrossValidationExperiment(SimpleExperiment): """ Performs a simple cross-validation experiment. Can output the results either in ARFF or CSV. """ def __init__(self, datasets, classifiers, classification=True, runs=10, folds=10, result=None): """ Initializes the experiment. :param datasets: the filenames of datasets to use in the experiment :type datasets: list :param classifiers: the Classifier objects to use in the experiment :type classifiers: list :param classification: whether to perform classification :type classification: bool :param runs: the number of runs to perform :type runs: int :param folds: the number folds to use for CV :type folds: int :param result: the filename of the file to store the results in :type result: str """ if runs < 1: raise Exception("Number of runs must be at least 1!") if folds < 2: raise Exception("Number of folds must be at least 2!") if len(datasets) == 0: raise Exception("No datasets provided!") if len(classifiers) == 0: raise Exception("No classifiers provided!") if result is None: raise Exception("No filename for results provided!") super(SimpleCrossValidationExperiment, self).__init__( classification=classification, runs=runs, datasets=datasets, classifiers=classifiers, result=result) self.folds = folds def configure_resultproducer(self): """ Configures and returns the ResultProducer and PropertyPath as tuple. :return: producer and property path :rtype: tuple """ rproducer = javabridge.make_instance("weka/experiment/CrossValidationResultProducer", "()V") javabridge.call(rproducer, "setNumFolds", "(I)V", self.folds) speval, classifier = self.configure_splitevaluator() javabridge.call(rproducer, "setSplitEvaluator", "(Lweka/experiment/SplitEvaluator;)V", speval) prop_path = javabridge.get_env().make_object_array( 2, javabridge.get_env().find_class("weka/experiment/PropertyNode")) cls = javabridge.get_env().find_class("weka/experiment/CrossValidationResultProducer") desc = javabridge.make_instance( "java/beans/PropertyDescriptor", "(Ljava/lang/String;Ljava/lang/Class;)V", "splitEvaluator", cls) node = javabridge.make_instance( "weka/experiment/PropertyNode", "(Ljava/lang/Object;Ljava/beans/PropertyDescriptor;Ljava/lang/Class;)V", speval, desc, cls) javabridge.get_env().set_object_array_element(prop_path, 0, node) cls = javabridge.get_env().get_object_class(speval) desc = javabridge.make_instance( "java/beans/PropertyDescriptor", "(Ljava/lang/String;Ljava/lang/Class;)V", "classifier", cls) node = javabridge.make_instance( "weka/experiment/PropertyNode", "(Ljava/lang/Object;Ljava/beans/PropertyDescriptor;Ljava/lang/Class;)V", javabridge.call(speval, "getClass", "()Ljava/lang/Class;"), desc, cls) javabridge.get_env().set_object_array_element(prop_path, 1, node) return rproducer, prop_path class SimpleRandomSplitExperiment(SimpleExperiment): """ Performs a simple random split experiment. Can output the results either in ARFF or CSV. """ def __init__(self, datasets, classifiers, classification=True, runs=10, percentage=66.6, preserve_order=False, result=None): """ Initializes the experiment. :param classification: whether to perform classification or regression :type classification: bool :param runs: the number of runs to perform :type runs: int :param percentage: the percentage to use for training :type percentage: float :param preserve_order: whether to preserve the order in the datasets :type preserve_order: bool :param datasets: the filenames of datasets to use in the experiment :type datasets: list :param classifiers: the Classifier objects to use in the experiment :type classifiers: list :param result: the filename of the file to store the results in :type result: str """ if runs < 1: raise Exception("Number of runs must be at least 1!") if percentage <= 0: raise Exception("Percentage for training must be >0!") if percentage >= 100: raise Exception("Percentage for training must be <100!") if len(datasets) == 0: raise Exception("No datasets provided!") if len(classifiers) == 0: raise Exception("No classifiers provided!") if result is None: raise Exception("No filename for results provided!") super(SimpleRandomSplitExperiment, self).__init__( classification=classification, runs=runs, datasets=datasets, classifiers=classifiers, result=result) self.percentage = percentage self.preserve_order = preserve_order def configure_resultproducer(self): """ Configures and returns the ResultProducer and PropertyPath as tuple. :return: producer and property path :rtype: tuple """ rproducer = javabridge.make_instance("weka/experiment/RandomSplitResultProducer", "()V") javabridge.call(rproducer, "setRandomizeData", "(Z)V", not self.preserve_order) javabridge.call(rproducer, "setTrainPercent", "(D)V", self.percentage) speval, classifier = self.configure_splitevaluator() javabridge.call(rproducer, "setSplitEvaluator", "(Lweka/experiment/SplitEvaluator;)V", speval) prop_path = javabridge.get_env().make_object_array( 2, javabridge.get_env().find_class("weka/experiment/PropertyNode")) cls = javabridge.get_env().find_class("weka/experiment/RandomSplitResultProducer") desc = javabridge.make_instance( "java/beans/PropertyDescriptor", "(Ljava/lang/String;Ljava/lang/Class;)V", "splitEvaluator", cls) node = javabridge.make_instance( "weka/experiment/PropertyNode", "(Ljava/lang/Object;Ljava/beans/PropertyDescriptor;Ljava/lang/Class;)V", speval, desc, cls) javabridge.get_env().set_object_array_element(prop_path, 0, node) cls = javabridge.get_env().get_object_class(speval) desc = javabridge.make_instance( "java/beans/PropertyDescriptor", "(Ljava/lang/String;Ljava/lang/Class;)V", "classifier", cls) node = javabridge.make_instance( "weka/experiment/PropertyNode", "(Ljava/lang/Object;Ljava/beans/PropertyDescriptor;Ljava/lang/Class;)V", javabridge.call(speval, "getClass", "()Ljava/lang/Class;"), desc, cls) javabridge.get_env().set_object_array_element(prop_path, 1, node) return rproducer, prop_path class ResultMatrix(OptionHandler): """ For generating results from an Experiment run. """ def __init__(self, classname="weka.experiment.ResultMatrixPlainText", jobject=None, options=None): """ Initializes the specified ResultMatrix using either the classname or the supplied JB_Object. :param classname: the classname of the ResultMatrix :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to use :type options: list """ if jobject is None: jobject = ResultMatrix.new_instance(classname) self.enforce_type(jobject, "weka.experiment.ResultMatrix") super(ResultMatrix, self).__init__(jobject=jobject, options=options) @property def rows(self): """ Returns the row count. :return: the count :rtype: int """ return javabridge.call(self.jobject, "getRowCount", "()I") @property def columns(self): """ Returns the column count. :return: the count :rtype: int """ return javabridge.call(self.jobject, "getColCount", "()I") def is_col_hidden(self, index): """ Returns whether the column is hidden. :param index: the 0-based column index :type index: int :return: true if hidden :rtype: bool """ return javabridge.call(self.jobject, "getColHidden", "(I)Z", index) def hide_col(self, index): """ Hides the column. :param index: the 0-based column index :type index: int """ javabridge.call(self.jobject, "setColHidden", "(IZ)V", index, True) def show_col(self, index): """ Shows the column. :param index: the 0-based column index :type index: int """ javabridge.call(self.jobject, "setColHidden", "(IZ)V", index, False) def is_row_hidden(self, index): """ Returns whether the row is hidden. :param index: the 0-based row index :type index: int :return: true if hidden :rtype: bool """ return javabridge.call(self.jobject, "getRowHidden", "(I)Z", index) def hide_row(self, index): """ Hides the row. :param index: the 0-based row index :type index: int """ javabridge.call(self.jobject, "setRowHidden", "(IZ)V", index, True) def show_row(self, index): """ Shows the row. :param index: the 0-based row index :type index: int """ javabridge.call(self.jobject, "setRowHidden", "(IZ)V", index, False) def get_row_name(self, index): """ Returns the row name. :param index: the 0-based row index :type index: int :return: the row name, None if invalid index :rtype: str """ return javabridge.call(self.jobject, "getRowName", "(I)Ljava/lang/String;", index) def set_row_name(self, index, name): """ Sets the row name. :param index: the 0-based row index :type index: int :param name: the name of the row :type name: str """ javabridge.call(self.jobject, "setRowName", "(ILjava/lang/String;)V", index, name) def get_col_name(self, index): """ Returns the column name. :param index: the 0-based row index :type index: int :return: the column name, None if invalid index :rtype: str """ return javabridge.call(self.jobject, "getColName", "(I)Ljava/lang/String;", index) def set_col_name(self, index, name): """ Sets the column name. :param index: the 0-based row index :type index: int :param name: the name of the column :type name: str """ javabridge.call(self.jobject, "setColName", "(ILjava/lang/String;)V", index, name) def get_mean(self, col, row): """ Returns the mean at this location (if valid location). :param col: the 0-based column index :type col: int :param row: the 0-based row index :type row: int :return: the mean :rtype: float """ return javabridge.call(self.jobject, "getMean", "(II)D", col, row) def set_mean(self, col, row, mean): """ Sets the mean at this location (if valid location). :param col: the 0-based column index :type col: int :param row: the 0-based row index :type row: int :param mean: the mean to set :type mean: float """ javabridge.call(self.jobject, "setMean", "(IID)V", col, row, mean) def get_stdev(self, col, row): """ Returns the standard deviation at this location (if valid location). :param col: the 0-based column index :type col: int :param row: the 0-based row index :type row: int :return: the standard deviation :rtype: float """ return javabridge.call(self.jobject, "getStdDev", "(II)D", col, row) def set_stdev(self, col, row, stdev): """ Sets the standard deviation at this location (if valid location). :param col: the 0-based column index :type col: int :param row: the 0-based row index :type row: int :param stdev: the standard deviation to set :type stdev: float """ javabridge.call(self.jobject, "setStdDev", "(IID)V", col, row, stdev) def average(self, col): """ Returns the average mean at this location (if valid location). :param col: the 0-based column index :type col: int :return: the mean :rtype: float """ return javabridge.call(self.jobject, "getAverage", "(I)D", col) def to_string_matrix(self): """ Returns the matrix as a string. :return: the generated output :rtype: str """ return javabridge.call(self.jobject, "toStringMatrix", "()V") def to_string_key(self): """ Returns a key for all the col names, for better readability if the names got cut off. :return: the key :rtype: str """ return javabridge.call(self.jobject, "toStringKey", "()V") def to_string_header(self): """ Returns the header of the matrix as a string. :return: the header :rtype: str """ return javabridge.call(self.jobject, "toStringHeader", "()V") def to_string_summary(self): """ returns the summary as string. :return: the summary :rtype: str """ return javabridge.call(self.jobject, "toStringSummary", "()V") def to_string_ranking(self): """ Returns the ranking in a string representation. :return: the ranking :rtype: str """ return javabridge.call(self.jobject, "toStringRanking", "()V") class Tester(OptionHandler): """ For generating statistical results from an experiment. """ def __init__(self, classname="weka.experiment.PairedCorrectedTTester", jobject=None, options=None): """ Initializes the specified tester using either the classname or the supplied JB_Object. :param classname: the classname of the tester :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Tester.new_instance(classname) self.enforce_type(jobject, "weka.experiment.Tester") self.columns_determined = False self._dataset_columns = ["Key_Dataset"] self._run_column = "Key_Run" self._fold_column = "Key_Fold" self._result_columns = ["Key_Scheme", "Key_Scheme_options", "Key_Scheme_version_ID"] super(Tester, self).__init__(jobject=jobject, options=options) @property def resultmatrix(self): """ Returns the ResultMatrix instance in use. :return: the matrix in use :rtype: ResultMatrix """ return ResultMatrix( jobject=javabridge.call(self.jobject, "getResultMatrix", "()Lweka/experiment/ResultMatrix;")) @resultmatrix.setter def resultmatrix(self, matrix): """ Sets the ResultMatrix to use. :param matrix: the ResultMatrix instance to use :type matrix: ResultMatrix """ javabridge.call(self.jobject, "setResultMatrix", "(Lweka/experiment/ResultMatrix;)V", matrix.jobject) @property def instances(self): """ Returns the data used in the analysis. :return: the data in use :rtype: Instances """ inst = javabridge.call(self.jobject, "getInstances", "()Lweka/core/Instances;") if inst is None: return None else: return Instances(inst) @instances.setter def instances(self, data): """ Sets the data to use for analysis. :param data: the Instances to analyze :type data: Instances """ javabridge.call(self.jobject, "setInstances", "(Lweka/core/Instances;)V", data.jobject) self.columns_determined = False @property def dataset_columns(self): """ Returns the list of column names that identify uniquely a dataset. :return: the list of attributes names :rtype: list """ return self._dataset_columns @dataset_columns.setter def dataset_columns(self, col_names): """ Sets the list of column names that identify uniquely a dataset. :param col_names: the list of attribute names :type col_names: list """ self._dataset_columns = col_names[:] @property def run_column(self): """ Returns the column name that holds the Run number. :return: the attribute name :rtype: str """ return self._run_column @run_column.setter def run_column(self, col_name): """ Sets the column name that holds the Run number. :param col_name: the attribute name :type col_name: str """ self._run_column = col_name @property def fold_column(self): """ Returns the column name that holds the Fold number. :return: the attribute name :rtype: str """ return self._fold_column @fold_column.setter def fold_column(self, col_name): """ Sets the column name that holds the Fold number. :param col_name: the attribute name :type col_name: str """ self._fold_column = col_name @property def result_columns(self): """ Returns the list of column names that identify uniquely a result (eg classifier + options + ID). :return: the list of attribute names :rtype: list """ return self._result_columns @result_columns.setter def result_columns(self, col_names): """ Sets the list of column names that identify uniquely a result (eg classifier + options + ID). :param col_names: the list of attribute names :type col_names: list """ self._result_columns = col_names[:] def init_columns(self): """ Sets the column indices based on the supplied names if necessary. """ if self.columns_determined: return data = self.instances if data is None: print("No instances set, cannot determine columns!") return # dataset if self.dataset_columns is None: raise Exception("No dataset columns set!") cols = "" for name in self.dataset_columns: att = data.attribute_by_name(name) if att is None: raise Exception("Dataset column not found: " + name) if len(cols) > 0: cols += "," cols += str(att.index + 1) javabridge.call( self.jobject, "setDatasetKeyColumns", "(Lweka/core/Range;)V", Range(ranges=cols).jobject) # run if self.run_column is None: raise Exception("No run columnn set!") att = data.attribute_by_name(self.run_column) if att is None: raise Exception("Run column not found: " + self.run_column) javabridge.call( self.jobject, "setRunColumn", "(I)V", att.index) # fold if not self.fold_column is None: att = data.attribute_by_name(self.fold_column) if att is None: index = -1 else: index = att.index javabridge.call( self.jobject, "setFoldColumn", "(I)V", index) # result if self._result_columns is None: raise Exception("No reset columns set!") cols = "" for name in self._result_columns: att = data.attribute_by_name(name) if att is None: raise Exception("Result column not found: " + name) if len(cols) > 0: cols += "," cols += str(att.index + 1) javabridge.call( self.jobject, "setResultsetKeyColumns", "(Lweka/core/Range;)V", Range(ranges=cols).jobject) self.columns_determined = True def header(self, comparison_column): """ Creates a "header" string describing the current resultsets. :param comparison_column: the index of the column to compare against :type comparison_column: int :return: the header :rtype: str """ self.init_columns() return javabridge.call( self.jobject, "header", "(I)Ljava/lang/String;", comparison_column) def multi_resultset_full(self, base_resultset, comparison_column): """ Creates a comparison table where a base resultset is compared to the other resultsets. :param base_resultset: the 0-based index of the base resultset (eg classifier to compare against) :type base_resultset: int :param comparison_column: the 0-based index of the column to compare against :type comparison_column: int :return: the comparison :rtype: str """ return javabridge.call( self.jobject, "multiResultsetFull", "(II)Ljava/lang/String;", base_resultset, comparison_column) def multi_resultset_ranking(self, comparison_column): """ Creates a ranking. :param comparison_column: the 0-based index of the column to compare against :type comparison_column: int :return: the ranking :rtype: str """ return javabridge.call( self.jobject, "multiResultsetRanking", "(I)Ljava/lang/String;", comparison_column) def multi_resultset_summary(self, comparison_column): """ Carries out a comparison between all resultsets, counting the number of datsets where one resultset outperforms the other. :param comparison_column: the 0-based index of the column to compare against :type comparison_column: int :return: the summary :rtype: str """ return javabridge.call( self.jobject, "multiResultsetSummary", "(I)Ljava/lang/String;", comparison_column) python-weka-wrapper3-0.1.3/python/weka/plot/0000775017267001726700000000000013147133421022401 5ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/python/weka/plot/clusterers.py0000644017267001726700000000716412712225162025155 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # clusterers.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import logging import weka.plot as plot if plot.matplotlib_available: import matplotlib.pyplot as plt from weka.core.dataset import Instances from weka.clusterers import ClusterEvaluation # logging setup logger = logging.getLogger(__name__) def plot_cluster_assignments(evl, data, atts=None, inst_no=False, size=10, title=None, outfile=None, wait=True): """ Plots the cluster assignments against the specified attributes. TODO: click events http://matplotlib.org/examples/event_handling/data_browser.html :param evl: the cluster evaluation to obtain the cluster assignments from :type evl: ClusterEvaluation :param data: the dataset the clusterer was evaluated against :type data: Instances :param atts: the list of attribute indices to plot, None for all :type atts: list :param inst_no: whether to include a fake attribute with the instance number :type inst_no: bool :param size: the size of the circles in point :type size: int :param title: an optional title :type title: str :param outfile: the (optional) file to save the generated plot to. The extension determines the file format. :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return fig = plt.figure() if data.class_index == -1: c = None else: c = [] for i in range(data.num_instances): inst = data.get_instance(i) c.append(inst.get_value(inst.class_index)) if atts is None: atts = [] for i in range(data.num_attributes): atts.append(i) num_plots = len(atts) if inst_no: num_plots += 1 clusters = evl.cluster_assignments for index, att in enumerate(atts): x = data.values(att) ax = fig.add_subplot( 1, num_plots, index + 1) if c is None: ax.scatter(clusters, x, s=size, alpha=0.5) else: ax.scatter(clusters, x, c=c, s=size, alpha=0.5) ax.set_xlabel("Clusters") ax.set_title(data.attribute(att).name) ax.get_xaxis().set_ticks(list(set(clusters))) ax.grid(True) if inst_no: x = [] for i in range(data.num_instances): x.append(i+1) ax = fig.add_subplot( 1, num_plots, num_plots) if c is None: ax.scatter(clusters, x, s=size, alpha=0.5) else: ax.scatter(clusters, x, c=c, s=size, alpha=0.5) ax.set_xlabel("Clusters") ax.set_title("Instance number") ax.get_xaxis().set_ticks(list(set(clusters))) ax.grid(True) if title is None: title = data.relationname fig.canvas.set_window_title(title) plt.draw() if not outfile is None: plt.savefig(outfile) if wait: plt.show() python-weka-wrapper3-0.1.3/python/weka/plot/classifiers.py0000644017267001726700000003276712764172000025277 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # classifiers.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import weka.plot as plot if plot.matplotlib_available: import matplotlib.pyplot as plt from weka.core.classes import JavaObject, join_options from weka.core.dataset import Instances from weka.classifiers import Classifier, Evaluation, NumericPrediction, NominalPrediction # logging setup logger = logging.getLogger(__name__) def plot_classifier_errors(predictions, absolute=True, max_relative_size=50, absolute_size=50, title=None, outfile=None, wait=True): """ Plots the classifers for the given list of predictions. TODO: click events http://matplotlib.org/examples/event_handling/data_browser.html :param predictions: the predictions to plot :type predictions: list :param absolute: whether to use absolute errors as size or relative ones :type absolute: bool :param max_relative_size: the maximum size in point in case of relative mode :type max_relative_size: int :param absolute_size: the size in point in case of absolute mode :type absolute_size: int :param title: an optional title :type title: str :param outfile: the output file, ignored if None :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return actual = [] predicted = [] error = None cls = None for pred in predictions: actual.append(pred.actual) predicted.append(pred.predicted) if isinstance(pred, NumericPrediction): if error is None: error = [] error.append(abs(pred.error)) elif isinstance(pred, NominalPrediction): if cls is None: cls = [] if pred.actual != pred.predicted: cls.append(1) else: cls.append(0) fig, ax = plt.subplots() if error is None and cls is None: ax.scatter(actual, predicted, s=absolute_size, alpha=0.5) elif cls is not None: ax.scatter(actual, predicted, c=cls, s=absolute_size, alpha=0.5) elif error is not None: if not absolute: min_err = min(error) max_err = max(error) factor = (max_err - min_err) / max_relative_size for i in range(len(error)): error[i] = error[i] / factor * max_relative_size ax.scatter(actual, predicted, s=error, alpha=0.5) ax.set_xlabel("actual") ax.set_ylabel("predicted") if title is None: title = "Classifier errors" ax.set_title(title) ax.plot(ax.get_xlim(), ax.get_ylim(), ls="--", c="0.3") ax.grid(True) fig.canvas.set_window_title(title) plt.draw() if outfile is not None: plt.savefig(outfile) if wait: plt.show() def generate_thresholdcurve_data(evaluation, class_index): """ Generates the threshold curve data from the evaluation object's predictions. :param evaluation: the evaluation to obtain the predictions from :type evaluation: Evaluation :param class_index: the 0-based index of the class-label to create the plot for :type class_index: int :return: the generated threshold curve data :rtype: Instances """ jtc = JavaObject.new_instance("weka.classifiers.evaluation.ThresholdCurve") pred = javabridge.call(evaluation.jobject, "predictions", "()Ljava/util/ArrayList;") result = Instances( javabridge.call(jtc, "getCurve", "(Ljava/util/ArrayList;I)Lweka/core/Instances;", pred, class_index)) return result def get_thresholdcurve_data(data, xname, yname): """ Retrieves x and y columns from of the data generated by the weka.classifiers.evaluation.ThresholdCurve class. :param data: the threshold curve data :type data: Instances :param xname: the name of the X column :type xname: str :param yname: the name of the Y column :type yname: str :return: tuple of x and y arrays :rtype: tuple """ xi = data.attribute_by_name(xname).index yi = data.attribute_by_name(yname).index x = [] y = [] for i in range(data.num_instances): inst = data.get_instance(i) x.append(inst.get_value(xi)) y.append(inst.get_value(yi)) return x, y def get_auc(data): """ Calculates the area under the ROC curve (AUC). :param data: the threshold curve data :type data: Instances :return: the area :rtype: float """ return javabridge.static_call( "weka/classifiers/evaluation/ThresholdCurve", "getROCArea", "(Lweka/core/Instances;)D", data.jobject) def get_prc(data): """ Calculates the area under the precision recall curve (PRC). :param data: the threshold curve data :type data: Instances :return: the area :rtype: float """ return javabridge.static_call( "weka/classifiers/evaluation/ThresholdCurve", "getPRCArea", "(Lweka/core/Instances;)D", data.jobject) def plot_roc(evaluation, class_index=None, title=None, key_loc="lower right", outfile=None, wait=True): """ Plots the ROC (receiver operator characteristics) curve for the given predictions. TODO: click events http://matplotlib.org/examples/event_handling/data_browser.html :param evaluation: the evaluation to obtain the predictions from :type evaluation: Evaluation :param class_index: the list of 0-based indices of the class-labels to create the plot for :type class_index: list :param title: an optional title :type title: str :param key_loc: the position string for the key :type key_loc: str :param outfile: the output file, ignored if None :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return if class_index is None: class_index = [0] ax = None for cindex in class_index: data = generate_thresholdcurve_data(evaluation, cindex) head = evaluation.header area = get_auc(data) x, y = get_thresholdcurve_data(data, "False Positive Rate", "True Positive Rate") if ax is None: fig, ax = plt.subplots() ax.set_xlabel("False Positive Rate") ax.set_ylabel("True Positive Rate") if title is None: title = "ROC" ax.set_title(title) ax.grid(True) fig.canvas.set_window_title(title) plt.xlim([-0.05, 1.05]) plt.ylim([-0.05, 1.05]) plot_label = head.class_attribute.value(cindex) + " (AUC: %0.4f)" % area ax.plot(x, y, label=plot_label) ax.plot(ax.get_xlim(), ax.get_ylim(), ls="--", c="0.3") plt.draw() plt.legend(loc=key_loc, shadow=True) if outfile is not None: plt.savefig(outfile) if wait: plt.show() def plot_prc(evaluation, class_index=None, title=None, key_loc="lower center", outfile=None, wait=True): """ Plots the PRC (precision recall) curve for the given predictions. TODO: click events http://matplotlib.org/examples/event_handling/data_browser.html :param evaluation: the evaluation to obtain the predictions from :type evaluation: Evaluation :param class_index: the list of 0-based indices of the class-labels to create the plot for :type class_index: list :param title: an optional title :type title: str :param key_loc: the location string for the key :type key_loc: str :param outfile: the output file, ignored if None :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return if class_index is None: class_index = [0] ax = None for cindex in class_index: data = generate_thresholdcurve_data(evaluation, cindex) head = evaluation.header area = get_prc(data) x, y = get_thresholdcurve_data(data, "Recall", "Precision") if ax is None: fig, ax = plt.subplots() ax.set_xlabel("Recall") ax.set_ylabel("Precision") if title is None: title = "PRC" ax.set_title(title) fig.canvas.set_window_title(title) plt.xlim([-0.05, 1.05]) plt.ylim([-0.05, 1.05]) ax.grid(True) plot_label = head.class_attribute.value(cindex) + " (PRC: %0.4f)" % area ax.plot(x, y, label=plot_label) ax.plot(ax.get_xlim(), ax.get_ylim(), ls="--", c="0.3") plt.draw() plt.legend(loc=key_loc, shadow=True) if outfile is not None: plt.savefig(outfile) if wait: plt.show() def plot_learning_curve(classifiers, train, test=None, increments=100, metric="percent_correct", title="Learning curve", label_template="[#] @ $", key_loc="lower right", outfile=None, wait=True): """ Plots a learning curve. :param classifiers: list of Classifier template objects :type classifiers: list of Classifier :param train: dataset to use for the building the classifier, used for evaluating it test set None :type train: Instances :param test: optional dataset (or list of datasets) to use for the testing the built classifiers :type test: list or Instances :param increments: the increments (>= 1: # of instances, <1: percentage of dataset) :type increments: float :param metric: the name of the numeric metric to plot (Evaluation.) :type metric: str :param title: the title for the plot :type title: str :param label_template: the template for the label in the plot (#: 1-based index of classifier, @: full classname, !: simple classname, $: options, *: 1-based index of test set) :type label_template: str :param key_loc: the location string for the key :type key_loc: str :param outfile: the output file, ignored if None :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return if not train.has_class(): logger.error("Training set has no class attribute set!") return if increments >= 1: inc = increments else: inc = round(train.num_instances * increments) if test is None: tst = [train] elif isinstance(test, list): tst = test elif isinstance(test, Instances): tst = [test] else: logger.error("Expected list or Instances object, instead: " + type(test)) return for t in tst: if train.equal_headers(t) is not None: logger.error("Training and test set are not compatible: " + train.equal_headers(t)) return steps = [] cls = [] evls = {} for classifier in classifiers: cl = Classifier.make_copy(classifier) cls.append(cl) evls[cl] = {} for t in tst: evls[cl][t] = [] for i in range(train.num_instances): if (i > 0) and (i % inc == 0): steps.append(i+1) for cl in cls: # train if cl.is_updateable: if i == 0: tr = Instances.copy_instances(train, 0, 1) cl.build_classifier(tr) else: cl.update_classifier(train.get_instance(i)) else: if (i > 0) and (i % inc == 0): tr = Instances.copy_instances(train, 0, i + 1) cl.build_classifier(tr) # evaluate if (i > 0) and (i % inc == 0): for t in tst: evl = Evaluation(t) evl.test_model(cl, t) evls[cl][t].append(getattr(evl, metric)) fig, ax = plt.subplots() ax.set_xlabel("# of instances") ax.set_ylabel(metric) ax.set_title(title) fig.canvas.set_window_title(title) ax.grid(True) i = 0 for cl in cls: evlpertest = evls[cl] i += 1 n = 0 for t in tst: evl = evlpertest[t] n += 1 plot_label = label_template.\ replace("#", str(i)).\ replace("*", str(n)).\ replace("@", cl.classname).\ replace("!", cl.classname[cl.classname.rfind(".") + 1:]).\ replace("$", join_options(cl.config)) ax.plot(steps, evl, label=plot_label) plt.draw() plt.legend(loc=key_loc, shadow=True) if outfile is not None: plt.savefig(outfile) if wait: plt.show() python-weka-wrapper3-0.1.3/python/weka/plot/dataset.py0000644017267001726700000001637713001477042024413 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # dataset.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import logging import weka.plot as plot if plot.matplotlib_available: import matplotlib.pyplot as plt from weka.core.dataset import Instances # logging setup logger = logging.getLogger(__name__) def scatter_plot(data, index_x, index_y, percent=100.0, seed=1, size=50, title=None, outfile=None, wait=True): """ Plots two attributes against each other. TODO: click events http://matplotlib.org/examples/event_handling/data_browser.html :param data: the dataset :type data: Instances :param index_x: the 0-based index of the attribute on the x axis :type index_x: int :param index_y: the 0-based index of the attribute on the y axis :type index_y: int :param percent: the percentage of the dataset to use for plotting :type percent: float :param seed: the seed value to use for subsampling :type seed: int :param size: the size of the circles in point :type size: int :param title: an optional title :type title: str :param outfile: the (optional) file to save the generated plot to. The extension determines the file format. :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return # create subsample data = plot.create_subsample(data, percent=percent, seed=seed) # collect data x = [] y = [] if data.class_index == -1: c = None else: c = [] for i in range(data.num_instances): inst = data.get_instance(i) x.append(inst.get_value(index_x)) y.append(inst.get_value(index_y)) if c is not None: c.append(inst.get_value(inst.class_index)) # plot data fig, ax = plt.subplots() if c is None: ax.scatter(x, y, s=size, alpha=0.5) else: ax.scatter(x, y, c=c, s=size, alpha=0.5) ax.set_xlabel(data.attribute(index_x).name) ax.set_ylabel(data.attribute(index_y).name) if title is None: title = "Attribute scatter plot" if percent != 100: title += " (%0.1f%%)" % percent ax.set_title(title) ax.plot(ax.get_xlim(), ax.get_ylim(), ls="--", c="0.3") ax.grid(True) fig.canvas.set_window_title(data.relationname) plt.draw() if outfile is not None: plt.savefig(outfile) if wait: plt.show() def matrix_plot(data, percent=100.0, seed=1, size=10, title=None, outfile=None, wait=True): """ Plots all attributes against each other. TODO: click events http://matplotlib.org/examples/event_handling/data_browser.html :param data: the dataset :type data: Instances :param percent: the percentage of the dataset to use for plotting :type percent: float :param seed: the seed value to use for subsampling :type seed: int :param size: the size of the circles in point :type size: int :param title: an optional title :type title: str :param outfile: the (optional) file to save the generated plot to. The extension determines the file format. :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return # create subsample data = plot.create_subsample(data, percent=percent, seed=seed) fig = plt.figure() if data.class_index == -1: c = None else: c = [] for i in range(data.num_instances): inst = data.get_instance(i) c.append(inst.get_value(inst.class_index)) for index_x in range(data.num_attributes): x = [] for i in range(data.num_instances): inst = data.get_instance(i) x.append(inst.get_value(index_x)) for index_y in range(data.num_attributes): y = [] for i in range(data.num_instances): inst = data.get_instance(i) y.append(inst.get_value(index_y)) ax = fig.add_subplot( data.num_attributes, data.num_attributes, index_x * data.num_attributes + index_y + 1) if c is None: ax.scatter(x, y, s=size, alpha=0.5) else: ax.scatter(x, y, c=c, s=size, alpha=0.5) ax.set_xlabel(data.attribute(index_x).name) ax.set_ylabel(data.attribute(index_y).name) ax.get_yaxis().set_ticklabels([]) ax.get_xaxis().set_ticklabels([]) ax.plot(ax.get_xlim(), ax.get_ylim(), ls="--", c="0.3") ax.grid(True) if title is None: title = data.relationname if percent != 100: title += " (%0.1f%%)" % percent fig.canvas.set_window_title(title) plt.draw() if outfile is not None: plt.savefig(outfile) if wait: plt.show() def line_plot(data, atts=None, percent=100.0, seed=1, title=None, outfile=None, wait=True): """ Uses the internal format to plot the dataset, one line per instance. :param data: the dataset :type data: Instances :param atts: the list of 0-based attribute indices of attributes to plot :type atts: list :param percent: the percentage of the dataset to use for plotting :type percent: float :param seed: the seed value to use for subsampling :type seed: int :param title: an optional title :type title: str :param outfile: the (optional) file to save the generated plot to. The extension determines the file format. :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return # create subsample data = plot.create_subsample(data, percent=percent, seed=seed) fig = plt.figure() if atts is None: x = [] for i in range(data.num_attributes): x.append(i) else: x = atts ax = fig.add_subplot(111) ax.set_xlabel("attributes") ax.set_ylabel("value") ax.grid(True) for index_y in range(data.num_instances): y = [] for index_x in x: y.append(data.get_instance(index_y).get_value(index_x)) ax.plot(x, y, "o-", alpha=0.5) if title is None: title = data.relationname if percent != 100: title += " (%0.1f%%)" % percent fig.canvas.set_window_title(title) plt.draw() if outfile is not None: plt.savefig(outfile) if wait: plt.show() python-weka-wrapper3-0.1.3/python/weka/plot/experiments.py0000644017267001726700000000643712712225170025326 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # experiments.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import logging import math import weka.plot as plot if plot.matplotlib_available: import matplotlib.pyplot as plt from weka.experiments import ResultMatrix # logging setup logger = logging.getLogger(__name__) def plot_experiment(mat, title="Experiment", axes_swapped=False, measure="Statistic", show_stdev=False, key_loc="lower right", outfile=None, wait=True): """ Plots the results from an experiment. :param mat: the result matrix to plot :type mat: ResultMatrix :param title: the title for the experiment :type title: str :param axes_swapped: whether the axes whether swapped ("sets x cls" or "cls x sets") :type axes_swapped: bool :param measure: the measure that is being displayed :type measure: str :param show_stdev: whether to show the standard deviation as error bar :type show_stdev: bool :param key_loc: the location string for the key :type key_loc: str :param outfile: the output file, ignored if None :type outfile: str :param wait: whether to wait for the user to close the plot :type wait: bool """ if not plot.matplotlib_available: logger.error("Matplotlib is not installed, plotting unavailable!") return if not isinstance(mat, ResultMatrix): logger.error("Need to supply a result matrix!") return fig, ax = plt.subplots() if axes_swapped: ax.set_xlabel(measure) ax.set_ylabel("Classifiers") else: ax.set_xlabel("Classifiers") ax.set_ylabel(measure) ax.set_title(title) fig.canvas.set_window_title(title) ax.grid(True) ticksx = [] ticks = [] inc = 1.0 / float(mat.columns) for i in range(mat.columns): ticksx.append((i + 0.5) * inc) ticks.append("[" + str(i+1) + "]") plt.xticks(ticksx, ticks) plt.xlim([0.0, 1.0]) for r in range(mat.rows): x = [] means = [] stdevs = [] for c in range(mat.columns): mean = mat.get_mean(c, r) stdev = mat.get_stdev(c, r) if not math.isnan(mean): x.append((c + 0.5) * inc) means.append(mean) if not math.isnan(stdev): stdevs.append(stdev) plot_label = mat.get_row_name(r) if show_stdev: ax.errorbar(x, means, yerr=stdevs, fmt='-o', ls="-", label=plot_label) else: ax.plot(x, means, "o-", label=plot_label) plt.draw() plt.legend(loc=key_loc, shadow=True) if outfile is not None: plt.savefig(outfile) if wait: plt.show() python-weka-wrapper3-0.1.3/python/weka/plot/__init__.py0000644017267001726700000000337212712217400024512 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # __init__.py # Copyright (C) 2014 Fracpete (pythonwekawrapper at gmail dot com) from weka.core.dataset import Instances from weka.core.classes import Random # check whether pygraphviz is there pygraphviz_available = False try: import pygraphviz pygraphviz_available = True except ImportError: pass # check whether PIL is there PIL_available = False try: import PIL PIL_available = True except ImportError: pass # check whether matplotlib is there matplotlib_available = False try: import matplotlib matplotlib_available = True except ImportError: pass def create_subsample(data, percent, seed=1): """ Generates a subsample of the dataset. :param data: the data to create the subsample from :type data: Instances :param percent: the percentage (0-100) :type percent: float :param seed: the seed value to use :type seed: int """ if percent <= 0 or percent >= 100: return data data = Instances.copy_instances(data) data.randomize(Random(seed)) data = Instances.copy_instances(data, 0, int(round(data.num_instances * percent / 100.0))) return data python-weka-wrapper3-0.1.3/python/weka/plot/graph.py0000644017267001726700000000326412712225172024061 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # graph.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import logging import tempfile import weka.plot as plot if plot.pygraphviz_available: from pygraphviz.agraph import AGraph if plot.PIL_available: from PIL import Image # logging setup logger = logging.getLogger(__name__) def plot_dot_graph(graph, filename=None): """ Plots a graph in graphviz dot notation. :param graph: the dot notation graph :type graph: str :param filename: the (optional) file to save the generated plot to. The extension determines the file format. :type filename: str """ if not plot.pygraphviz_available: logger.error("Pygraphviz is not installed, cannot generate graph plot!") return if not plot.PIL_available: logger.error("PIL is not installed, cannot display graph plot!") return agraph = AGraph(graph) agraph.layout(prog='dot') if filename is None: filename = tempfile.mktemp(suffix=".png") agraph.draw(filename) image = Image.open(filename) image.show() python-weka-wrapper3-0.1.3/python/weka/__init__.py0000644017267001726700000000000012712217400023515 0ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/python/weka/associations.py0000644017267001726700000001060112712225032024465 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # associations.py # Copyright (C) 2014-2016 Fracpete (pythonwekawrapper at gmail dot com) import javabridge import logging import os import sys import argparse import weka.core.jvm as jvm import weka.core.converters as converters from weka.core.classes import OptionHandler, join_options from weka.core.capabilities import Capabilities # logging setup logger = logging.getLogger("weka.associations") class Associator(OptionHandler): """ Wrapper class for associators. """ def __init__(self, classname=None, jobject=None, options=None): """ Initializes the specified associator using either the classname or the supplied JB_Object. :param classname: the classname of the associator :type classname: str :param jobject: the JB_Object to use :type jobject: JB_Object :param options: the list of commandline options to set :type options: list """ if jobject is None: jobject = Associator.new_instance(classname) self.enforce_type(jobject, "weka.associations.Associator") super(Associator, self).__init__(jobject=jobject, options=options) @property def capabilities(self): """ Returns the capabilities of the associator. :return: the capabilities :rtype: Capabilities """ return Capabilities(javabridge.call(self.jobject, "getCapabilities", "()Lweka/core/Capabilities;")) def build_associations(self, data): """ Builds the associator with the data. :param data: the data to train the associator with :type data: Instances """ javabridge.call(self.jobject, "buildAssociations", "(Lweka/core/Instances;)V", data.jobject) @classmethod def make_copy(cls, associator): """ Creates a copy of the clusterer. :param associator: the associator to copy :type associator: Associator :return: the copy of the associator :rtype: Associator """ return Associator( jobject=javabridge.static_call( "weka/associations/AbstractAssociator", "makeCopy", "(Lweka/associations/Associator;)Lweka/associations/Associator;", associator.jobject)) def main(): """ Runs a associator from the command-line. Calls JVM start/stop automatically. Use -h to see all options. """ parser = argparse.ArgumentParser( description='Executes an associator from the command-line. Calls JVM start/stop automatically.') parser.add_argument("-j", metavar="classpath", dest="classpath", help="additional classpath, jars/directories") parser.add_argument("-X", metavar="heap", dest="heap", help="max heap size for jvm, e.g., 512m") parser.add_argument("-t", metavar="train", dest="train", required=True, help="training set file") parser.add_argument("associator", help="associator classname, e.g., weka.associations.Apriori") parser.add_argument("option", nargs=argparse.REMAINDER, help="additional associator options") parsed = parser.parse_args() jars = [] if parsed.classpath is not None: jars = parsed.classpath.split(os.pathsep) jvm.start(jars, max_heap_size=parsed.heap, packages=True) logger.debug("Commandline: " + join_options(sys.argv[1:])) try: associator = Associator(classname=parsed.associator) if len(parsed.option) > 0: associator.options = parsed.option loader = converters.loader_for_file(parsed.train) data = loader.load_file(parsed.train) associator.build_associations(data) print(str(associator)) except Exception as e: print(e) finally: jvm.stop() if __name__ == "__main__": try: main() except Exception as ex: print(ex) python-weka-wrapper3-0.1.3/python/weka/lib/0000775017267001726700000000000013147133421022171 5ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/python/weka/lib/python-weka-wrapper-src.jar0000644017267001726700000000611412712217400027375 0ustar fracpetefracpete00000000000000PKÇu_E META-INF/þÊPKPKÇu_EMETA-INF/MANIFEST.MFóMÌËLK-.Ñ K-*ÎÌϳR0Ô3àår.JM,IMÑuª ˜ëÄ›˜*hø%&ç¤*8çä%–•kòrñrPKXECDPK 蚎Dweka/PK Š]_Eweka/classifiers/PKDu_E"weka/classifiers/KernelHelper.javaíWMsÛ6=K¿b›K©Ô¡’6‡ŽU§qT+QãØInÆ“ú‘K 1pPª&ñï %Ú¢l§“ô” %»ûñv·á1Lf\C®dªØèo¢AËÄ,™Â}XÉ"&@a̵Q|Zn€‰¸+ÌeÌ“•Å¡±BĨÀÌ ª¹™¸—×'çð*–ÁY1ÍxÇ ÆhÃB pG‰×%*cŒ†ñLW‰_Pc5E—Å0c ¤GȃHæ«û›gAX&EêÒ$ãM!)¸aBš=Ðäo3còýnw¹\†©(B©ÒnæAt÷… ¨Ûnw]ToQ ÌÞ`–£ ?²³ƒ}ŠGñtf èwàç§ÏžCX”#Q1HªÔŸtNB, ¥0ïxÜœEW,EXâ £ŒiÍNèµÛÔ © ØeÂ)2¡ÃWôŠDöæ†Â(©sŒŒTMógŠx¢ÌêÔ‘âù–UÆD*L2Bß!1#ÞÄp;º0)Dd©«Ãñ»ÓÞÃ̦±ÔEnMþr™„¾â_î7z5¸ÏÕŒuøçó_o™ÃsC\ìÙ¶»¾ûŽƒóv”½rØk¶¾dL¹¾Ÿ=¤ï¹çª¬³ >µÛnY àšB í¸$\iÓ)׶dv ­¤ §mëÀ9zï—9³ûÍÔ,ÀHHѸ‘ÛP‰’óÒU¹•›¬ö@´Ëð¤Œ%jñ£ñB•B—že–ÚÐöÁ6m>ùj.8õ„Ü¡RPƒ[«¥P™é¹ùJ­V‹;m4;}¸lÙ´ŸïÛ“G”•îáaáÀ%X7jU`×2©ë-¤<ª0 ݾ»‚N§Wú¹ÅÉÏúÛùíɺ2¶ êeAö=”-O‹zöÕ"¾WÜ WnПE>†µÉY¼eÑi·€˜) öÚ¶&g“‹·¼˜¬r´áºœÃAIc/1]­ ­gG¦ Ùý».3õ¿þ1Í 8ú'Bw.vÖeoh 1+mp¢Ra®¸0™ HktLÉi͵ް;Þ”zÿѺÖè}džvâ ©«6\׉áÄ°¡Ýõ •ög]iXÎÐÿV3´è‚ÇH©‘Ý%XY"°;ò:¬ð!‚ì ·”© ´"¬úEnÉu‡§Rf6 ò¹W‚åŠÁ¶\Ñ%‘¶ 3F£k[ȃRÕ7÷¦Ä^™jåpÍÂûì.Ni¶¤âXl]D–t=Êî«®ò­Õd¯j¢‡­JTºÜÜš¶./;éEÕj-­l›6¡„eº>a]i¸¹ ÞΪљU-YëÇoa~±šºoì½r>äb!¯0pÙ \‚ÏþÃå'Ÿóµµ\‡iK]i©Ru³’[wëÕ6Ç/àÔÙÚ)Ì›»Æ.º™5_71¾ì˜¼ëtÜŠÉ:!9¥+­ØÁ±’EiŪ®ù‰ÿ@)Úcãk_ƒPv)€çE&ÁÛR/ÖcÞ^v¾ •ÒoI¥Ä5—)Ýv²U—˜Ju‹n{sú͸@7O€§‚®—ñN,$.‹"ÓWžH?\:¿ñLÉ¥†Mê©èyà*¹¾øV)ö×W^°·ØdêzëD·s2_O”Å¥oÃMqÿOž¼€Gð“u«]pìß6ÇõU§ÉÙL>ÊßÄö¡C w”ÍtÎînÙ÷å=¦ònN&tÒf—ñ¡èºç^µ«õP¶ NŸCßkþÖœ¾ô¾üüº}ÝþPK†®„̵PK/[¶D'weka/classifiers/EvaluationWrapper.javaÕT]oÛ6}÷¯8È“Òúñ”lC½Àn ¤‰a;ò4ÐÒ•ÅV"’²êù､$Ûi¶¶Ø°‡½¤xïáù¸t|>À9Ö¹´¨ŒÞQ‚—™!‚Õ™k„¡ ìuD(J¥uFnjGB¥±6(u*³½ÇáoµJÉÀåG¦´ÐYؼ½¹Ã[RdDE½)d‚k™²ÁWû/6§›€ã;fžÃªã€™f`á¤VcäsƒË{¼êïèÇÐƃ …óÌ tåûFLwB¸ckÄe%ÿ¨2…T;×+Ê’56²(°!Ô–²º{.Æý|ýîönÉÍî'ËåäfýpÉÅ.×|J;j¡dY’‘Y—Ê홾Gx?]^½ã–ÉóëùúE`6_ßLW+Ìn—˜`1Y®çWwד%wËÅíj+ò´È|Çâ,¤Ä6¦ä„,l/üƒµÌ®H‘‹qÀ ÉsHtµÿqxDZmƒL.>Éäæ”vcX&ùkî\uÇMÓD[UGÚl㢱ñïžP<ÄÕt'Š:d}oDU‘‰>ˆð'WLÊÈmî0¼áå//^󘈤"žÇaÖ¯8¤mÉ2‘jÇ:ÊQ ^‰ä£Øú(¢¤ÖÊLò \‰6®;aŸ¢¹²N¨„üa|Xu\:©z{Î\ntc1ý”Pµ3_ÄÃÑO¢ÎÔ‰Ó¦wþ¨Àú‹Ÿ¡_µ¾··?s_€™†ûšð1å‘ëË 7(q‘G‚OÈñŽ+Ž@(ÿ BG‡\ŃHH#†$:ƒè’ÄA&ºƒè‘DRAJÅ|iÐR–³-XmZ¦»VÀ®Þ-àk´†@YÌ´Œ–ÞT‡áìÔ;’Ü™Ðe¸Û Ç2’;;m8î1eáX·~X$u«+²½£Ûˆ»õÕù­C·2‘‚è#wÌ´k;õãÚ\=ÞÓ¬§s–鳂´‚C F†Ô3#ªë }%™(¦1dbGŒ=OꙌÙiN&ÒÙkÅ]Ó&¶Lo:m;înÊÙN$o…F™ Úí²»qL'Õ0#›ÒMK`r¸½È^›ë˜VW½Œ¼ºñhÜHçLh³{¸ÑdæBŸ·±ÙH2‹¤²†…¨Õ« WÃaa5Å1ÆRÃK8®áeWðŠ†Wá D¯ÀÄ!w)xMÃë8!° I'´DȵCiÇŽ™ŒœsÙ %î&;Uð††“8¥áM 8‚Åå#éá4Î0ÉC6™ÖPOHtØ~WñþÒ‡ö©µ¡©àÖ¬û(µ5o×ð–toÊÈ 5ôšÉ„áøÕººµ! gqNÁy pQÃÛ¸( ±nÁ­£ÀÓX)¥. ss5Ö(xGÃe¼K…!/¥Â*Ч9÷‡ç]ÓFɸŽad"[—¯˜9ŠY\<äß÷vÒžç7pÜbu²I„Úx«ƒÌM£<–6ÂÕE%žÛ¬/Ä4oa‹å:v&ËŽ@)U‡Î ‡G)ë)IšœLÅÑ`ˆ`n¸úÁˆÇS{cºF³á´3†ávŒÎ$Fò\Ù·(ßj艂øÔ{ÉDÝ^ÇÚy,m +ù¶c×H±Ý]¯ÌhÚ„hZ.‹ÍÐSõ…4 ßPÒ’JZ#bQ)KçåÙ`w:zœGLë°ÝÃŪ1úXû=ûwñÁÚ½l"=6,ºW7Ö飶ï ÕËF$”ž0@-zŠ°&…GkÊrríüZ¦eß}ç+ÞvŒÈ.×LÒø8×n´S)ÝJÈWL`þ½Ÿ©âC1‡dä_BvXŽu¤frœý57!¾çB`Ñ øŽ–ȶ–³á! ±š,<ýð6×þŸ@K]Ô_忽"à]¡T*•+¨¯òW*K£Áª`þ;˜/çÀoôazn™…õUù&"x%UÁXJN)¢oà×>LŠúEìÔŠÒ,´*ã~G´c?÷äà5aǤVAE”T=ý[ÍÕþÖòÛd¶c=Zñ v‘×Îo”ýül9@Ír»±6¶a ­lF`3T°TÁ2Ë<þ Fà_„¸÷¦ Àß0 ðtO~ BÁŠr¢ccŒëâÁ¸Ž«éÇø;P|}ðy¯qÃSß]9~xJ×ó]–]rPû ¤‡Þ5R¿¥îjëJ´ßO(Ûë]Ø–Eùßš>TDý2^ ñšqVJ1óîÆki Aª“®u‘2³n„ÐC¹4%SX‹É¶±œO¯„5ž2ò^׋‚›ãi”­~èex‰Xš ÐæÚ;XPË·ÔÕx “†LoEÔ'1VHŒ¾BNKY}¡»k˜Ià©“ÄxŠÜÓ(ÇÊ\ ÔYââUÿ˜åþ ËéS\Å\ÇgøŸão|!‚¸*æ¡OlÀ—"†¯D¾gpM\Å7â\â[¶°”ä%l)ò7üPKYkH PKÇu_E(weka/classifiers/EvaluationWrapper.class…’]OA†ßi Cë"壀B¥Ti·¥eAåB«øA Ñ´hR¢ñÊL—‘,´»ÍîQ/ü9ÞªQšhâðG©g¶Å–ØàÅÎì¼gæ=Ïœ9?}ûà:ÖcÐä¸C Q\@Šc1Š4®ŽàZ KÈÄ0„¬ ë9Ž<ƒÖx¹u$ê-á[ŽÍ,¿–‡Â0ëÂó¬W–t=£.1 ß±lËß`˜Õ»;Wl϶)½RîCdÓÙ“ ceË–;­FMº»¢V'e´ê ó°"šÁ: X&B†õÁfÿ²l:ž_¾kS&ŽyîK¿ÿ zî?WˆU–kÊmK!M÷BÏ]ÑlJwå@ c(2Ì㤶Ä5Œâ"ä:cÔ…½ol›²©â+ ¬2d·åÚKùNJÏuë­LõœRNí@šþmŽ5žñ††›(*û†ô9]`†x/ý“ÀŠˆÔsÐ}zeëêõú̪o<_6( ]Ê’(Ë1žº–íW}WŠFéôògeÞT«:½HB/÷YR*{?h“±fg?µÄ®+L‰EêX:™:H••þÂôOU¦qœVïHӜɷÁòßz±ÜFø+"¨|Æð?ÁHáÑ<#ñSàÇ$µ|t.uRó¤¯R¤€)1¤è©–°Fè ýVÇGŒcz<™é‚¼'ËÍÙH¡X‡d甤x–dœr%¹Ej‰ô¹K‰6Ä=ªÂ}dð0 ™V$SwH8fÿÂ\Âå.L’fFóPþ "ƒò©Ãø˜Æ9šC˜ÿPKÇNcY'PKÇu_E META-INF/þÊPKÇu_EXECD=META-INF/MANIFEST.MFPK Çu_EÂweka/PK Çu_Eåweka/classifiers/PKÇu_EYkH #weka/classifiers/KernelHelper.classPKÇu_EÇNcY'(weka/classifiers/EvaluationWrapper.classPK–0 python-weka-wrapper3-0.1.3/python/weka/flow/0000775017267001726700000000000013147133421022372 5ustar fracpetefracpete00000000000000python-weka-wrapper3-0.1.3/python/weka/flow/container.py0000644017267001726700000001737212712225136024740 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # container.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import re from weka.core.dataset import Instances class Container(object): """ Container for storing multiple objects and passing them around together in the flow. """ def __init__(self): """ Initializes the container. """ self._data = {} self._allowed = [] def get(self, name): """ Returns the stored data. :param name: the name of the item to return :type name: str :return: the data :rtype: object """ return self._data[name] def set(self, name, value): """ Stores the given data (if not None). :param name: the name of the item to store :type name: str :param value: the value to store :type value: object """ if value is not None: self._data[name] = value @property def allowed(self): """ Returns the all the allowed keys. :return: the list of allowed keys. :rtype: list """ return self._allowed def is_valid(self): """ Checks whether the container is valid. :return: True if the container is valid :rtype: bool """ return True def __str__(self): """ Returns the content of the container as string. :return: the content :rtype: str """ return str(self._data) def generate_help(self): """ Generates a help string for this container. :return: the help string :rtype: str """ result = [] result.append(self.__class__.__name__) result.append(re.sub(r'.', '=', self.__class__.__name__)) result.append("") result.append("Supported value names:") for a in self.allowed: result.append(a) return '\n'.join(result) def print_help(self): """ Prints a help string for this actor to stdout. """ print(self.generate_help()) class ModelContainer(Container): """ Container for models. """ def __init__(self, model=None, header=None): """ Initializes the container. :param model: the model to store (eg Classifier or Clusterer) :type model: object :param header: the header instances :type header: Instances """ super(ModelContainer, self).__init__() self.set("Model", model) if header is not None: header = Instances.template_instances(header) self.set("Header", header) self._allowed = ["Model", "Header"] def is_valid(self): """ Checks whether the container is valid. :return: True if the container is valid :rtype: bool """ return ("Model" in self._data) or ("Model" in self._data and "Header" in self._data) class AttributeSelectionContainer(Container): """ Container for models. """ def __init__(self, original=None, reduced=None, num_atts=None, selected=None, results=None): """ Initializes the container. :param original: the original dataset :type original: Instances :param reduced: the reduced dataset :type reduced: Instances :param num_atts: the number of attributes :type num_atts: int :param selected: the list of selected attribute indices (0-based) :type selected: list :param results: the generated results string :type results: str """ super(AttributeSelectionContainer, self).__init__() self.set("Original", original) self.set("Reduced", reduced) self.set("NumAttributes", num_atts) self.set("Selected", selected) self.set("Results", results) self._allowed = ["Original", "Reduced", "NumAttributes", "Selected", "Results"] def is_valid(self): """ Checks whether the container is valid. :return: True if the container is valid :rtype: bool """ return ("Reduced" in self._data) and ("NumAttributes" in self._data) and ("Selected" in self._data) class ModelContainer(Container): """ Container for models. """ def __init__(self, model=None, header=None): """ Initializes the container. :param model: the model to store (eg Classifier or Clusterer) :type model: object :param header: the header instances :type header: Instances """ super(ModelContainer, self).__init__() self.set("Model", model) if header is not None: header = Instances.template_instances(header) self.set("Header", header) self._allowed = ["Model", "Header"] def is_valid(self): """ Checks whether the container is valid. :return: True if the container is valid :rtype: bool """ return ("Model" in self._data) or ("Model" in self._data and "Header" in self._data) class ClassificationContainer(Container): """ Container for predictions (classifiers). """ def __init__(self, inst=None, classification=None, label=None, distribution=None): """ Initializes the container. :param inst: the instance used for making the prediction :type inst: Instance :param classification: the classification (numeric value or 0-based label index) :type classification: float :param label: classification label (for nominal classes) :type label: str :param distribution: the class distribution :type distribution: ndarray """ super(ClassificationContainer, self).__init__() self.set("Instance", inst) self.set("Classification", classification) self.set("Label", label) self.set("Distribution", distribution) self._allowed = ["Instance", "Classification", "Label", "Distribution"] def is_valid(self): """ Checks whether the container is valid. :return: True if the container is valid :rtype: bool """ return ("Instance" in self._data) and ("Classification" in self._data) class ClusteringContainer(Container): """ Container for predictions (clusterers). """ def __init__(self, inst=None, cluster=None, distribution=None): """ Initializes the container. :param inst: the instance used for making the prediction :type inst: Instance :param cluster: the cluster :type cluster: int :param distribution: the class distribution :type distribution: ndarray """ super(ClusteringContainer, self).__init__() self.set("Instance", inst) self.set("Cluster", cluster) self.set("Distribution", distribution) self._allowed = ["Instance", "Cluster", "Distribution"] def is_valid(self): """ Checks whether the container is valid. :return: True if the container is valid :rtype: bool """ return ("Instance" in self._data) and ("Cluster" in self._data) python-weka-wrapper3-0.1.3/python/weka/flow/sink.py0000644017267001726700000007203412712225150023712 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # sink.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import traceback import weka.core.serialization as serialization from weka.core.dataset import Instances, Instance from weka.flow.base import InputConsumer from weka.flow.container import ModelContainer from weka.classifiers import Evaluation import weka.plot.classifiers as pltclassifier import weka.plot.clusterers as pltclusterer import weka.plot.dataset as pltdataset class Sink(InputConsumer): """ The ancestor for all sinks. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Sink, self).__init__(name=name, config=config) super(InputConsumer, self).__init__(name=name, config=config) def post_execute(self): """ Gets executed after the actual execution. :return: None if successful, otherwise error message :rtype: str """ result = super(Sink, self).post_execute() if result is None: self._input = None return result class Null(Sink): """ Sink that just gobbles up all the data. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Null, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Sink that just gobbles up all the data." def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ return None class Console(Sink): """ Sink that outputs the payloads of the data on stdout. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Console, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Sink that outputs the payloads of the data on stdout." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "prefix: '" + str(self.config["prefix"]) + "'" def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(Console, self).fix_config(options) opt = "prefix" if opt not in options: options[opt] = "" if opt not in self.help: self.help[opt] = "The prefix for the output (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ print(self.resolve_option("prefix") + str(self.input.payload)) return None class FileOutputSink(Sink): """ Ancestor for sinks that output data to a file. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(FileOutputSink, self).__init__(name=name, config=config) @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "output: '" + str(self.config["output"]) + "'" def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(FileOutputSink, self).fix_config(options) opt = "output" if opt not in options: options[opt] = "." if opt not in self.help: self.help[opt] = "The file to write to (string)." return options class DumpFile(FileOutputSink): """ Sink that outputs the payloads of the data to a file. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(DumpFile, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Sink that outputs the payloads of the data to a file." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return super(DumpFile, self).quickinfo + ", append: " + str(self.config["append"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(DumpFile, self).fix_config(options) opt = "append" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to append to the file or overwrite (bool)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None f = None try: if bool(self.resolve_option("append")): f = open(str(self.resolve_option("output")), "a") else: f = open(str(self.resolve_option("output")), "w") f.write(str(self.input.payload)) f.write("\n") except Exception as e: result = self.full_name + "\n" + traceback.format_exc() finally: if f is not None: f.close() return result class ModelWriter(FileOutputSink): """ Writes a model to disk. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ModelWriter, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Writes a model to disk." def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if not isinstance(token.payload, ModelContainer): raise Exception(self.full_name + ": Input token is not a ModelContainer!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None cont = self.input.payload serialization.write_all( str(self.resolve_option("output")), [cont.get("Model").jobject, cont.get("Header").jobject]) return result class MatrixPlot(Sink): """ Displays the Instances object as matrix plot. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(MatrixPlot, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Displays the Instances object as matrix plot." def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(MatrixPlot, self).fix_config(options) opt = "percent" if opt not in options: options[opt] = 100.0 if opt not in self.help: self.help[opt] = "The percentage of the data to display (0-100, float)." opt = "seed" if opt not in options: options[opt] = 1 if opt not in self.help: self.help[opt] = "The seed value for randomizing the plot when viewing a subset (int)." opt = "size" if opt not in options: options[opt] = 10 if opt not in self.help: self.help[opt] = "The size of the circles in the plot (int)." opt = "title" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The title for the plot (str)." opt = "outfile" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The file to store the plot in (str)." opt = "wait" if opt not in options: options[opt] = True if opt not in self.help: self.help[opt] = "Whether to wait for user to close the plot window (bool)." return options @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "percent: " + str(self.config["percent"]) \ + ", title: " + str(self.config["title"]) \ + ", outfile: " + str(self.config["outfile"]) \ + ", wait: " + str(self.config["wait"]) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if not isinstance(token.payload, Instances): raise Exception(self.full_name + ": Input token is not an Instances object!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None data = self.input.payload pltdataset.matrix_plot( data, percent=float(self.resolve_option("percent")), seed=int(self.resolve_option("seed")), size=int(self.resolve_option("size")), title=self.resolve_option("title"), outfile=self.resolve_option("outfile"), wait=bool(self.resolve_option("wait"))) return result class LinePlot(Sink): """ Displays the Instances object as line plot using the internal format, one line per Instance. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(LinePlot, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Displays the Instances object as line plot using the internal format, one line per Instance." def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(LinePlot, self).fix_config(options) opt = "attributes" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The list of 0-based attribute indices to print; None for all (int)." opt = "percent" if opt not in options: options[opt] = 100.0 if opt not in self.help: self.help[opt] = "The percentage of the data to display (0-100, float)." opt = "seed" if opt not in options: options[opt] = 1 if opt not in self.help: self.help[opt] = "The seed value for randomizing the plot when viewing a subset (int)." opt = "title" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The title for the plot (str)." opt = "outfile" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The file to store the plot in (str)." opt = "wait" if opt not in options: options[opt] = True if opt not in self.help: self.help[opt] = "Whether to wait for user to close the plot window (bool)." return options @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "percent: " + str(self.config["percent"]) \ + ", title: " + str(self.config["title"]) \ + ", outfile: " + str(self.config["outfile"]) \ + ", wait: " + str(self.config["wait"]) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if not isinstance(token.payload, Instances): raise Exception(self.full_name + ": Input token is not an Instances object!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None data = self.input.payload pltdataset.line_plot( data, atts=self.resolve_option("attributes"), percent=float(self.resolve_option("percent")), seed=int(self.resolve_option("seed")), title=self.resolve_option("title"), outfile=self.resolve_option("outfile"), wait=bool(self.resolve_option("wait"))) return result class ClassifierErrors(Sink): """ Displays the errors obtained through a classifier evaluation. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ClassifierErrors, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Displays the errors obtained through a classifier evaluation." def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(ClassifierErrors, self).fix_config(options) opt = "absolute" if opt not in options: options[opt] = True if opt not in self.help: self.help[opt] = "Whether to use absolute errors as size or relative ones (bool)." opt = "max_relative_size" if opt not in options: options[opt] = 50 if opt not in self.help: self.help[opt] = "The maximum size in point in case of relative mode (int)." opt = "absolute_size" if opt not in options: options[opt] = 50 if opt not in self.help: self.help[opt] = "The size in point in case of absolute mode (int)." opt = "title" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The title for the plot (str)." opt = "outfile" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The file to store the plot in (str)." opt = "wait" if opt not in options: options[opt] = True if opt not in self.help: self.help[opt] = "Whether to wait for user to close the plot window (bool)." return options @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "absolute: " + str(self.config["absolute"]) \ + ", title: " + str(self.config["title"]) \ + ", outfile: " + str(self.config["outfile"]) \ + ", wait: " + str(self.config["wait"]) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if not isinstance(token.payload, Evaluation): raise Exception(self.full_name + ": Input token is not an Evaluation object!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None evl = self.input.payload pltclassifier.plot_classifier_errors( evl.predictions, absolute=bool(self.resolve_option("absolute")), max_relative_size=int(self.resolve_option("max_relative_size")), absolute_size=int(self.resolve_option("absolute_size")), title=self.resolve_option("title"), outfile=self.resolve_option("outfile"), wait=bool(self.resolve_option("wait"))) return result class ROC(Sink): """ Displays the ROC curve obtained from a classifier evaluation. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ROC, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Displays the ROC curve obtained from a classifier evaluation." def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(ROC, self).fix_config(options) opt = "class_index" if opt not in options: options[opt] = [0] if opt not in self.help: self.help[opt] = "The list of 0-based class-label indices to display (list)." opt = "key_loc" if opt not in options: options[opt] = "lower right" if opt not in self.help: self.help[opt] = "The location of the key in the plot (str)." opt = "title" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The title for the plot (str)." opt = "outfile" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The file to store the plot in (str)." opt = "wait" if opt not in options: options[opt] = True if opt not in self.help: self.help[opt] = "Whether to wait for user to close the plot window (bool)." return options @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "classes: " + str(self.config["class_index"]) \ + ", title: " + str(self.config["title"]) \ + ", outfile: " + str(self.config["outfile"]) \ + ", wait: " + str(self.config["wait"]) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if not isinstance(token.payload, Evaluation): raise Exception(self.full_name + ": Input token is not an Evaluation object!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None evl = self.input.payload pltclassifier.plot_roc( evl, class_index=self.resolve_option("class_index"), title=self.resolve_option("title"), key_loc=self.resolve_option("key_loc"), outfile=self.resolve_option("outfile"), wait=bool(self.resolve_option("wait"))) return result class PRC(Sink): """ Displays the PRC curve obtained from a classifier evaluation. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(PRC, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Displays the PRC curve obtained from a classifier evaluation." def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(PRC, self).fix_config(options) opt = "class_index" if opt not in options: options[opt] = [0] if opt not in self.help: self.help[opt] = "The list of 0-based class-label indices to display (list)." opt = "key_loc" if opt not in options: options[opt] = "lower center" if opt not in self.help: self.help[opt] = "The location of the key in the plot (str)." opt = "title" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The title for the plot (str)." opt = "outfile" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The file to store the plot in (str)." opt = "wait" if opt not in options: options[opt] = True if opt not in self.help: self.help[opt] = "Whether to wait for user to close the plot window (bool)." return options @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "classes: " + str(self.config["class_index"]) \ + ", title: " + str(self.config["title"]) \ + ", outfile: " + str(self.config["outfile"]) \ + ", wait: " + str(self.config["wait"]) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if not isinstance(token.payload, Evaluation): raise Exception(self.full_name + ": Input token is not an Evaluation object!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None evl = self.input.payload pltclassifier.plot_prc( evl, class_index=self.resolve_option("class_index"), title=self.resolve_option("title"), key_loc=self.resolve_option("key_loc"), outfile=self.resolve_option("outfile"), wait=bool(self.resolve_option("wait"))) return result class InstanceDumper(FileOutputSink): """ Sink that dumps the incoming Instance/Instances into a file. """ def __init__(self, name=None, config=None): """ Initializes the sink. :param name: the name of the sink :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(InstanceDumper, self).__init__(name=name, config=config) self._header = None def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Sink that dumps the incoming Instance/Instances objects in a file." def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if isinstance(token.payload, Instance): return if isinstance(token.payload, Instances): return raise Exception(self.full_name + ": Input token is neither an Instance nor Instances object!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None data = self.input.payload if isinstance(self._input.payload, Instance): inst = self.input.payload data = inst.dataset elif isinstance(self.input.payload, Instances): data = self.input.payload inst = None append = True if self._header is None or (self._header.equal_headers(data) is not None): self._header = Instances.template_instances(data, 0) outstr = str(data) append = False elif inst is not None: outstr = str(inst) else: outstr = str(data) f = None try: if append: f = open(str(self.resolve_option("output")), "a") else: f = open(str(self.resolve_option("output")), "w") f.write(outstr) f.write("\n") except Exception as e: result = self.full_name + "\n" + traceback.format_exc() finally: if f is not None: f.close() return result python-weka-wrapper3-0.1.3/python/weka/flow/base.py0000644017267001726700000004166712712225127023674 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # base.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import logging import re import traceback import uuid from weka.core.classes import Configurable, Stoppable import weka.core.classes as classes class Actor(Configurable, Stoppable): """ The ancestor for all actors. """ def __init__(self, name=None, config=None): """ Initializes the actor. :param name: the name of the actor :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Actor, self).__init__(config=config) self._name = self.__class__.__name__ self._parent = None self._full_name = None self._stopped = False if name is not None: self.name = name if not classes.has_dict_handler("Actor"): classes.register_dict_handler("Actor", Actor.from_dict) def __str__(self): """ Returns a short representation of the actor's setup. :return: the setup :rtype: str """ return self.full_name + ": " + str(self._config) def __repr__(self): """ Returns Python code for instantiating the object. :return: the representation :rtype: str """ return \ self.__class__.__module__ + "." + self.__class__.__name__ \ + "(name=" + self.name + ", config=" + str(self.config) + ")" def new_logger(self): """ Returns a new logger instance. :return: the logger instance :rtype: logger """ return logging.getLogger(self.full_name) @property def name(self): """ Obtains the currently set name of the actor. :return: the name :rtype: str """ return self._name @name.setter def name(self, name): """ Sets the name of the actor. :param name: the name :type name: str """ self._name = name def unique_name(self, name): """ Generates a unique name. :param name: the name to check :type name: str :return: the unique name :rtype: str """ result = name if self.parent is not None: index = self.index bname = re.sub(r'-[0-9]+$', '', name) names = [] for idx, actor in enumerate(self.parent.actors): if idx != index: names.append(actor.name) result = bname count = 0 while result in names: count += 1 result = bname + "-" + str(count) return result @property def parent(self): """ Obtains the currently set parent of the actor. :return: the name :rtype: str """ return self._parent @parent.setter def parent(self, parent): """ Sets the parent of the actor. :param parent: the parent :type parent: Actor """ self._name = self.unique_name(self._name) self._full_name = None self._logger = None self._parent = parent @property def index(self): """ Returns the index of this actor in its parent's list of actors. :return: the index, -1 if not available :rtype: int """ if self.parent is None: return -1 else: return self.parent.index_of(self.name) @property def full_name(self): """ Obtains the full name of the actor. :return: the full name :rtype: str """ if self._full_name is None: fn = self.name.replace(".", "\\.") parent = self._parent if parent is not None: fn = parent.full_name + "." + fn self._full_name = fn return self._full_name def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ opt = "annotation" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The (optional) annotation for this actor (string)." opt = "skip" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to skip (disable) this actor (bool)." return super(Actor, self).fix_config(options) def to_dict(self): """ Returns a dictionary that represents this object, to be used for JSONification. :return: the object dictionary :rtype: dict """ result = super(Actor, self).to_dict() result["type"] = "Actor" result["name"] = self.name return result @classmethod def from_dict(cls, d): """ Restores an object state from a dictionary, used in de-JSONification. :param d: the object dictionary :type d: dict :return: the object :rtype: object """ conf = {} for k in d["config"]: v = d["config"][k] if isinstance(v, dict): if u"type" in v: typestr = v[u"type"] else: typestr = v["type"] conf[str(k)] = classes.get_dict_handler(typestr)(v) else: conf[str(k)] = v return classes.get_class(d["class"])(name=d["name"], config=conf) def resolve_option(self, name, default=None): """ Resolves the option, i.e., interprets "@{...}" values and retrievs them instead from internal storage. :param name: the name of the option :type name: str :param default: the optional default value :type default: object :return: the resolved value :rtype: object """ value = self.config[name] if value is None: return default elif isinstance(value, str) \ and value.startswith("@{") \ and value.endswith("}") \ and (value.find("@{", 1) == -1): stname = value[2:len(value)-1] if (self.storagehandler is not None) and (stname in self.storagehandler.storage): return self.storagehandler.storage[stname] else: return default else: return value @property def skip(self): """ Obtains whether the actor is disabled (skipped). :return: True if skipped :rtype: bool """ return self.resolve_option("skip") @skip.setter def skip(self, skip): """ Sets whether the actor is skipped. :param skip: True if skipped :type skip: bool """ self.config["skip"] = skip @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return None @property def storagehandler(self): """ Returns the storage handler available to thise actor. :return: the storage handler, None if not available """ if isinstance(self, StorageHandler): return self elif self.parent is not None: return self.parent.storagehandler else: return None @property def root(self): """ Returns the top-level actor. :return: the top-level actor :rtype: Actor """ if self.parent is None: return self else: return self.parent.root @property def depth(self): """ Returns the depth of this actor inside the overall flow. :return: the depth :rtype: int """ if self.parent is None: return 0 else: return self.parent.depth + 1 def is_stopped(self): """ Returns whether the object has been stopped. :return: whether stopped :rtype: bool """ return self._stopped def stop_execution(self): """ Triggers the stopping of the actor. """ self._stopped = True def setup(self): """ Configures the actor before execution. :return: None if successful, otherwise error message :rtype: str """ return None def pre_execute(self): """ Gets executed before the actual execution. :return: None if successful, otherwise error message :rtype: str """ return None def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ raise Exception("Not implemented!") def post_execute(self): """ Gets executed after the actual execution. :return: None if successful, otherwise error message :rtype: str """ return None def execute(self): """ Executes the actor. :return: None if successful, otherwise error message :rtype: str """ if self.skip: return None result = self.pre_execute() if result is None: try: result = self.do_execute() except Exception as e: result = traceback.format_exc() print(self.full_name + "\n" + result) if result is None: result = self.post_execute() return result def wrapup(self): """ Finishes up after execution finishes, does not remove any graphical output. """ pass def cleanup(self): """ Destructive finishing up after execution stopped. """ pass class Token(object): """ Container for transporting data through the flow. """ def __init__(self, payload): """ Initializes the token with the given payload. :param payload: the payload for the token. :type payload: object """ self._id = str(uuid.uuid4()) self._payload = payload @property def id(self): """ Obtains the ID of the token. :return: the ID :rtype: str """ return self._id @property def payload(self): """ Obtains the currently set payload. :return: the payload :rtype: object """ return self._payload def __str__(self): """ Returns a short representation of the token and its payload. """ return self._id + ": " + str(self._payload) class InputConsumer(Actor): """ Actors that consume tokens inherit this class. """ def __init__(self, name=None, config=None): """ Initializes the actor. :param name: the name of the actor :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(InputConsumer, self).__init__(name=name, config=config) self._input = None def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ pass @property def input(self): """ Returns the current input token, None if not available. :return: the input token :rtype: Token """ return self._input @input.setter def input(self, token): """ Accepts the data for processing. :param token: the token to process :type token: Token """ self.check_input(token) self._input = token class OutputProducer(Actor): """ Actors that generate output tokens inherit this class. """ def __init__(self, name=None, config=None): """ Initializes the actor. :param name: the name of the actor :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(OutputProducer, self).__init__(name=name, config=config) self._output = None def pre_execute(self): """ Gets executed before the actual execution. :return: None if successful, otherwise error message :rtype: str """ self._output = [] return None def has_output(self): """ Checks whether any output tokens are present. :return: true if at least one output token present :rtype: bool """ return (self._output is not None) and (len(self._output) > 0) def output(self): """ Returns the next available output token. :return: the next token, None if none available :rtype: Token """ if (self._output is None) or (len(self._output) == 0): result = None else: result = self._output.pop(0) return result class StorageHandler(object): """ For classes that support internal storage (= dictionary). """ @property def storage(self): """ Returns the internal storage. :return: the internal storage :rtype: dict """ raise Exception("Not implemented!") def expand(self, s): """ Expands all occurrences of "@{...}" within the string with the actual values currently stored in internal storage. :param s: the string to expand :type s: str :return: the expanded string :rtype: str """ result = s while result.find("@{") > -1: start = result.index("@{") end = result.index("}", start) name = result[start + 2:end] value = self.storage[name] if value is None: raise("Storage value '" + name + "' not present, failed to expand string: " + s) else: result = result[0:start] + str(value) + result[end + 1:] return result @classmethod def pad(cls, name): """ Pads the name with "@{...}". :param name: the name to pad :type name: str :return: the padded name :rtype: str """ if name.startswith("@{"): return name else: return "@{" + name + "}" @classmethod def extract(cls, padded): """ Removes the surrounding "@{...}" from the name. :param padded: the padded string :type padded: str :return: the extracted name :rtype: str """ if padded.startswith("@{") and padded.endswith("}"): return padded[2:len(padded)-1] else: return padded def is_source(actor): """ Checks whether the actor is a source. :param actor: the actor to check :type actor: Actor :return: True if the actor is a source :rtype: bool """ return not isinstance(actor, InputConsumer) and isinstance(actor, OutputProducer) def is_transformer(actor): """ Checks whether the actor is a transformer. :param actor: the actor to check :type actor: Actor :return: True if the actor is a transformer :rtype: bool """ return isinstance(actor, InputConsumer) and isinstance(actor, OutputProducer) def is_sink(actor): """ Checks whether the actor is a sink. :param actor: the actor to check :type actor: Actor :return: True if the actor is a sink :rtype: bool """ return isinstance(actor, InputConsumer) and not isinstance(actor, OutputProducer) def to_commandline(o): """ Turns the object into a commandline string. However, first checks whether a string represents a internal value placeholder (@{...}). :param o: the object to turn into commandline :type o: object :return: the commandline :rtype: str """ if isinstance(o, str) and o.startswith("@{") and o.endswith("}"): return o else: return classes.to_commandline(o) python-weka-wrapper3-0.1.3/python/weka/flow/source.py0000644017267001726700000005577712712225153024270 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # source.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import os import re from weka.core.classes import to_commandline, from_commandline import weka.flow.base as base from weka.flow.base import Actor, OutputProducer, Token from weka.core.database import InstanceQuery import weka.datagenerators as datagen class Source(OutputProducer, Actor): """ The ancestor for all sources. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Source, self).__init__(name=name, config=config) super(OutputProducer, self).__init__(name=name, config=config) class Start(Source): """ Outputs a None token for triggering other actors. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Start, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Outputs a None token for triggering other actors." def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ self._output.append(Token(None)) return None class FileSupplier(Source): """ Outputs a fixed list of files. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(FileSupplier, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Outputs a fixed list of files." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "files: " + str(len(self.config["files"])) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(FileSupplier, self).fix_config(options) opt = "files" if opt not in options: options[opt] = [] if opt not in self.help: self.help[opt] = "The files to output (list of string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ for f in self.resolve_option("files"): self._output.append(Token(f)) return None class ListFiles(Source): """ Source that list files in a directory. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ListFiles, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Source that list files in a directory." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "dir: " + str(self.config["dir"]) \ + ", files: " + str(self.config["list_files"]) \ + ", dirs: " + str(self.resolve_option("list_dirs")) \ + ", recursive: " + str(self.config["recursive"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(ListFiles, self).fix_config(options) opt = "dir" if opt not in options: options[opt] = "." if opt not in self.help: self.help[opt] = "The directory to search (string)." opt = "recursive" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to search recursively (bool)." opt = "list_files" if opt not in options: options[opt] = True if opt not in self.help: self.help[opt] = "Whether to include files (bool)." opt = "list_dirs" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to include directories (bool)." opt = "regexp" if opt not in options: options[opt] = ".*" if opt not in self.help: self.help[opt] = "The regular expression that files/dirs must match (string)." return options def _list(self, path, collected): """ Lists all the files/dirs in directory that match the pattern. :param path: the directory to search :type path: str :param collected: the files/dirs collected so far (full path) :type collected: list :return: None if successful, error otherwise :rtype: str """ list_files = self.resolve_option("list_files") list_dirs = self.resolve_option("list_dirs") recursive = self.resolve_option("recursive") spattern = str(self.resolve_option("regexp")) pattern = None if (spattern is not None) and (spattern != ".*"): pattern = re.compile(spattern) try: items = os.listdir(path) for item in items: fp = path + os.sep + item if list_files and os.path.isfile(fp): if (pattern is None) or pattern.match(item): collected.append(fp) if list_dirs and os.path.isdir(fp): if (pattern is None) or pattern.match(item): collected.append(fp) if recursive and os.path.isdir(fp): self._list(fp, collected) except Exception as e: return "Error listing '" + path + "': " + str(e) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ directory = str(self.resolve_option("dir")) if not os.path.exists(directory): return "Directory '" + directory + "' does not exist!" if not os.path.isdir(directory): return "Location '" + directory + "' is not a directory!" collected = [] result = self._list(directory, collected) if result is None: for c in collected: self._output.append(Token(c)) return result class GetStorageValue(Source): """ Outputs the specified value from storage. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(GetStorageValue, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Outputs the specified value from storage." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "name: " + str(self.config["storage_name"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(GetStorageValue, self).fix_config(options) opt = "storage_name" if opt not in options: options[opt] = "unknown" if opt not in self.help: self.help[opt] = "The name of the storage value to retrieve (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if self.storagehandler is None: return "No storage handler available!" sname = str(self.resolve_option("storage_name")) if sname not in self.storagehandler.storage: return "No storage item called '" + sname + "' present!" self._output.append(Token(self.storagehandler.storage[sname])) return None class ForLoop(Source): """ Outputs integers using the specified min, max and step. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ForLoop, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Outputs integers using the specified min, max and step." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "min: " + str(self.config["min"]) \ + ", max: " + str(self.config["max"]) \ + ", step: " + str(self.config["step"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(ForLoop, self).fix_config(options) opt = "min" if opt not in options: options[opt] = 1 if opt not in self.help: self.help[opt] = "The minimum for the loop (included, int)." opt = "max" if opt not in options: options[opt] = 10 if opt not in self.help: self.help[opt] = "The maximum for the loop (included, int)." opt = "step" if opt not in options: options[opt] = 1 if opt not in self.help: self.help[opt] = "The step size (int)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ for i in range( int(self.resolve_option("min")), int(self.resolve_option("max")) + 1, int(self.resolve_option("step"))): self._output.append(Token(i)) return None class LoadDatabase(Source): """ Loads a dataset from the database. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(LoadDatabase, self).__init__(name=name, config=config) self._loader = None self._iterator = None def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Loads a dataset from a database using a supplied SQL query." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "url: " + str(self.config["db_url"]) \ + ", query: " + str(self.config["query"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ opt = "db_url" if opt not in options: options[opt] = "jdbc:mysql://somehost:3306/somedatabase" if opt not in self.help: self.help[opt] = "The JDBC database URL to connect to (str)." opt = "user" if opt not in options: options[opt] = "user" if opt not in self.help: self.help[opt] = "The database user to use for connecting (str)." opt = "password" if opt not in options: options[opt] = "secret" if opt not in self.help: self.help[opt] = "The password for the database user (str)." opt = "query" if opt not in options: options[opt] = "SELECT * FROM table" if opt not in self.help: self.help[opt] = "The SQL query for generating the dataset (str)." opt = "sparse" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to return the data in sparse format (bool)." opt = "custom_props" if opt not in options: options[opt] = "" if opt not in self.help: self.help[opt] = "Custom properties filename (str)." return super(LoadDatabase, self).fix_config(options) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ iquery = InstanceQuery() iquery.db_url = str(self.resolve_option("db_url")) iquery.user = str(self.resolve_option("user")) iquery.password = str(self.resolve_option("password")) props = str(self.resolve_option("custom_props")) if (len(props) > 0) and os.path.isfile(props): iquery.custom_properties = props iquery.query = str(self.resolve_option("query")) data = iquery.retrieve_instances() self._output.append(Token(data)) return None class DataGenerator(Source): """ Generates artificial data using the specified data generator. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(DataGenerator, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Generates artificial data using the specified data generator." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "setup: " + base.to_commandline(self.config["setup"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ opt = "setup" if opt not in options: options[opt] = datagen.DataGenerator(classname="weka.datagenerators.classifiers.classification.Agrawal") if opt not in self.help: self.help[opt] = "The data generator to use (DataGenerator)." opt = "incremental" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to output the data incrementally, in case the generator supports that (bool)." return super(DataGenerator, self).fix_config(options) def to_config(self, k, v): """ Hook method that allows conversion of individual options. :param k: the key of the option :type k: str :param v: the value :type v: object :return: the potentially processed value :rtype: object """ if k == "setup": return base.to_commandline(v) return super(DataGenerator, self).to_config(k, v) def from_config(self, k, v): """ Hook method that allows converting values from the dictionary. :param k: the key in the dictionary :type k: str :param v: the value :type v: object :return: the potentially parsed value :rtype: object """ if k == "setup": return from_commandline(v, classname=to_commandline(datagen.DataGenerator())) return super(DataGenerator, self).from_config(k, v) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ generator = datagen.DataGenerator.make_copy(self.resolve_option("setup")) generator.dataset_format = generator.define_data_format() if bool(self.resolve_option("incremental")) and generator.single_mode_flag: for i in range(generator.num_examples_act): self._output.append(Token(generator.generate_example())) else: data = generator.generate_examples() self._output.append(Token(data)) return None class CombineStorage(Source): """ Expands the storage items specified in format string and forwards the generated string. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(CombineStorage, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Expands the storage items specified in format string and forwards the generated string." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "format: " + str(self.config["format"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(CombineStorage, self).fix_config(options) opt = "format" if opt not in options: options[opt] = "" if opt not in self.help: self.help[opt] = "The format to use for generating the combined string; use '@{blah}' for accessing "\ "storage item 'blah' (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ formatstr = str(self.resolve_option("format")) expanded = self.storagehandler.expand(formatstr) self._output.append(Token(expanded)) return None class StringConstants(Source): """ Outputs a fixed list of strings. """ def __init__(self, name=None, config=None): """ Initializes the source. :param name: the name of the source :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(StringConstants, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Outputs a fixed list of strings." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "strings: " + str(len(self.config["strings"])) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(StringConstants, self).fix_config(options) opt = "strings" if opt not in options: options[opt] = [] if opt not in self.help: self.help[opt] = "The strings to output (list of string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ for s in self.resolve_option("strings"): self._output.append(Token(s)) return None python-weka-wrapper3-0.1.3/python/weka/flow/transformer.py0000644017267001726700000016642312712225155025323 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # transformer.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import os import re import javabridge import math # required for MathExpression from weka.associations import Associator import weka.core.classes as classes import weka.core.serialization as serialization import weka.attribute_selection as attsel import weka.filters as filters import weka.flow.base as base from weka.flow.base import InputConsumer, OutputProducer, Token from weka.flow.container import ModelContainer, AttributeSelectionContainer, ClassificationContainer, ClusteringContainer import weka.core.converters as converters from weka.core.dataset import Instances, Instance from weka.classifiers import Classifier, Evaluation from weka.clusterers import Clusterer, ClusterEvaluation from weka.core.classes import Random import weka.flow.conversion as conversion class Transformer(InputConsumer, OutputProducer): """ The ancestor for all sources. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(InputConsumer, self).__init__(name=name, config=config) super(OutputProducer, self).__init__(name=name, config=config) def post_execute(self): """ Gets executed after the actual execution. :return: None if successful, otherwise error message :rtype: str """ result = super(Transformer, self).post_execute() if result is None: self._input = None return result class PassThrough(Transformer): """ Dummy actor that just passes through the data. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(PassThrough, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Dummy actor that just passes through the data." def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ self._output.append(self.input) class LoadDataset(Transformer): """ Loads a dataset from a file. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(LoadDataset, self).__init__(name=name, config=config) self._loader = None self._iterator = None def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Loads a dataset from a file. Either all at once or incrementally." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "incremental: " + str(self.config["incremental"]) \ + ", custom: " + str(self.config["use_custom_loader"]) \ + ", loader: " + base.to_commandline(self.config["custom_loader"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ opt = "incremental" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to load the dataset incrementally (bool)." opt = "use_custom_loader" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to use a custom loader." opt = "custom_loader" if opt not in options: options[opt] = converters.Loader(classname="weka.core.converters.ArffLoader") if opt not in self.help: self.help[opt] = "The custom loader to use (Loader)." return super(LoadDataset, self).fix_config(options) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if token is None: raise Exception(self.full_name + ": No token provided!") if isinstance(token.payload, str): return raise Exception(self.full_name + ": Unhandled class: " + classes.get_classname(token.payload)) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ fname = str(self.input.payload) if not os.path.exists(fname): return "File '" + fname + "' does not exist!" if not os.path.isfile(fname): return "Location '" + fname + "' is not a file!" if self.resolve_option("use_custom_loader"): self._loader = self.resolve_option("custom_loader") else: self._loader = converters.loader_for_file(fname) dataset = self._loader.load_file(fname, incremental=bool(self.resolve_option("incremental"))) if not self.resolve_option("incremental"): self._output.append(Token(dataset)) else: self._iterator = self._loader.__iter__() return None def has_output(self): """ Checks whether any output tokens are present. :return: true if at least one output token present :rtype: bool """ return super(LoadDataset, self).has_output() or (self._iterator is not None) def output(self): """ Returns the next available output token. :return: the next token, None if none available :rtype: Token """ if self._iterator is not None: try: inst = self._iterator.next() result = Token(inst) except Exception as e: self._iterator = None result = None else: result = super(LoadDataset, self).output() return result def stop_execution(self): """ Triggers the stopping of the object. """ super(LoadDataset, self).stop_execution() self._loader = None self._iterator = None def wrapup(self): """ Finishes up after execution finishes, does not remove any graphical output. """ self._loader = None self._iterator = None super(LoadDataset, self).wrapup() class SetStorageValue(Transformer): """ Store the payload of the current token in internal storage using the specified name. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(SetStorageValue, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Store the payload of the current token in internal storage using the specified name." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "name: " + str(self.config["storage_name"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(SetStorageValue, self).fix_config(options) opt = "storage_name" if opt not in options: options[opt] = "unknown" if opt not in self.help: self.help[opt] = "The storage value name for storing the payload under (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if self.storagehandler is None: return "No storage handler available!" self.storagehandler.storage[self.resolve_option("storage_name")] = self.input.payload self._output.append(self.input) return None class DeleteStorageValue(Transformer): """ Deletes the specified value from internal storage. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(DeleteStorageValue, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Deletes the specified value from internal storage." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "name: " + str(self.config["storage_name"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(DeleteStorageValue, self).fix_config(options) opt = "storage_name" if opt not in options: options[opt] = "unknown" if opt not in self.help: self.help[opt] = "The name of the storage value to delete (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if self.storagehandler is None: return "No storage handler available!" self.storagehandler.storage.pop(self.resolve_option("storage_name"), None) self._output.append(self.input) return None class InitStorageValue(Transformer): """ Initializes the storage value with the provided value (interpreted by 'eval' method). """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(InitStorageValue, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Initializes the storage value with the provided value (interpreted by 'eval' method)." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "name: " + str(self.config["storage_name"]) + ", value: " + str(self.config["value"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(InitStorageValue, self).fix_config(options) opt = "storage_name" if opt not in options: options[opt] = "unknown" if opt not in self.help: self.help[opt] = "The name of the storage value to delete (string)." opt = "value" if opt not in options: options[opt] = "1" if opt not in self.help: self.help[opt] = "The initial value (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if self.storagehandler is None: return "No storage handler available!" self.storagehandler.storage[self.resolve_option("storage_name")] = eval(str(self.resolve_option("value"))) self._output.append(self.input) return None class UpdateStorageValue(Transformer): """ Updates the specified storage value using the epxression interpreted by 'eval' method. The current value is available through the variable {X} in the expression. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(UpdateStorageValue, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Updates the specified storage value using the epxression interpreted by 'eval' method.\n"\ "The current value is available through the variable {X} in the expression.\n"\ "Any storage value can be referenced using @{name} with 'name' being the name of the storage value." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "name: " + str(self.config["storage_name"]) + ", expression: " + str(self.config["expression"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(UpdateStorageValue, self).fix_config(options) opt = "storage_name" if opt not in options: options[opt] = "unknown" if opt not in self.help: self.help[opt] = "The name of the storage value to update (string)." opt = "expression" if opt not in options: options[opt] = "int({X} + 1)" if opt not in self.help: self.help[opt] = "The expression for updating the storage value; use {X} for current value (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if self.storagehandler is None: return "No storage handler available!" expr = str(self.resolve_option("expression")).replace( "{X}", str(self.storagehandler.storage[str(self.resolve_option("storage_name"))])) expr = self.storagehandler.expand(expr) self.storagehandler.storage[self.resolve_option("storage_name")] = eval(expr) self._output.append(self.input) return None class MathExpression(Transformer): """ Calculates a mathematical expression. The placeholder {X} in the expression gets replaced by the value of the current token passing through. Uses the 'eval(str)' method for the calculation, therefore mathematical functions can be accessed using the 'math' library, e.g., '1 + math.sin({X})'. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(MathExpression, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return \ "Calculates a mathematical expression. The placeholder {X} in the expression gets replaced by "\ + "the value of the current token passing through. Uses the 'eval(str)' method for the calculation, "\ + "therefore mathematical functions can be accessed using the 'math' library, e.g., '1 + math.sin({X})'." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "expression: " + str(self.config["expression"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(MathExpression, self).fix_config(options) opt = "expression" if opt not in options: options[opt] = "{X}" if opt not in self.help: self.help[opt] = "The mathematical expression to evaluate (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ expr = str(self.resolve_option("expression")) expr = expr.replace("{X}", str(self.input.payload)) self._output.append(Token(eval(expr))) return None class ClassSelector(Transformer): """ Sets/unsets the class index of a dataset. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ClassSelector, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Sets/unsets the class index of a dataset." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "index: " + str(self.config["index"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(ClassSelector, self).fix_config(options) opt = "index" if opt not in options: options[opt] = "last" if opt not in self.help: self.help[opt] = "The class index (1-based number); 'first' and 'last' are accepted as well (string)." opt = "unset" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to unset the class index (bool)." return options def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if isinstance(token.payload, Instances): return if isinstance(token.payload, Instance): return raise Exception(self.full_name + ": Unhandled data type: " + str(token.payload.__class__.__name__)) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if isinstance(self.input.payload, Instances): inst = None data = self.input.payload elif isinstance(self.input.payload, Instance): inst = self.input.payload data = inst.dataset index = str(self.resolve_option("index")) unset = bool(self.resolve_option("unset")) if unset: data.no_class() else: if index == "first": data.class_is_first() elif index == "last": data.class_is_last() else: data.class_index = int(index) - 1 if inst is None: self._output.append(Token(data)) else: self._output.append(Token(inst)) return None class Train(Transformer): """ Trains the classifier/clusterer/associator on the incoming dataset and forwards a ModelContainer with the trained model and the dataset header. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Train, self).__init__(name=name, config=config) self._model = None self._header = None def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return \ "Trains the classifier/clusterer/associator on the incoming dataset and forwards a ModelContainer with the trained " \ + "model and the dataset header." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "setup: " + base.to_commandline(self.config["setup"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(Train, self).fix_config(options) opt = "setup" if opt not in options: options[opt] = Classifier(classname="weka.classifiers.rules.ZeroR") if opt not in self.help: self.help[opt] = "The classifier/clusterer/associator to train (Classifier/Clusterer/Associator)." return options def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if isinstance(token.payload, Instances): return if isinstance(token.payload, Instance): return raise Exception(self.full_name + ": Unhandled data type: " + str(token.payload.__class__.__name__)) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if isinstance(self.input.payload, Instances): inst = None data = self.input.payload else: inst = self.input.payload data = inst.dataset retrain = False if (self._header is None) or (self._header.equal_headers(data) is not None) or (inst is None): retrain = True self._header = Instances.template_instances(data, 0) if retrain or (self._model is None): cls = self.resolve_option("setup") if isinstance(cls, Classifier): self._model = Classifier.make_copy(cls) elif isinstance(cls, Clusterer): self._model = Clusterer.make_copy(cls) elif isinstance(cls, Associator): self._model = Associator.make_copy(cls) else: return "Unhandled class: " + classes.get_classname(cls) if retrain: if inst is not None: data = Instances.template_instances(data, 1) data.add_instance(inst) if isinstance(self._model, Classifier): self._model.build_classifier(data) elif isinstance(self._model, Clusterer): self._model.build_clusterer(data) elif isinstance(self._model, Associator): self._model.build_associations(data) else: if isinstance(self._model, Classifier): self._model.update_classifier(inst) elif isinstance(self._model, Clusterer): self._model.update_clusterer(inst) else: return "Cannot train incrementally: " + classes.get_classname(self._model) cont = ModelContainer(model=self._model, header=self._header) self._output.append(Token(cont)) return None class Filter(Transformer): """ Filters a dataset with the specified filter setup. Automatically resets the filter if the dataset differs. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Filter, self).__init__(name=name, config=config) self._filter = None self._header = None def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Filters a dataset with the specified filter setup.\n"\ "Automatically resets the filter if the dataset differs." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "setup: " + base.to_commandline(self.config["setup"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ opt = "setup" if opt not in options: options[opt] = filters.Filter(classname="weka.filters.AllFilter") if opt not in self.help: self.help[opt] = "The filter to apply to the dataset (Filter)." opt = "keep_relationname" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to keep the original relation name (bool)." return super(Filter, self).fix_config(options) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if token is None: raise Exception(self.full_name + ": No token provided!") if isinstance(token.payload, Instances): return if isinstance(token.payload, Instance): return raise Exception(self.full_name + ": Unhandled class: " + classes.get_classname(token.payload)) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if isinstance(self.input.payload, Instance): inst = self.input.payload data = Instances.template_instances(inst.dataset, 1) data.add_instance(inst) else: inst = None data = self.input.payload relname = data.relationname keep = self.resolve_option("keep_relationname") if inst is None: if (self._filter is None) or self._header.equal_headers(data) is not None: self._header = Instances.template_instances(data) self._filter = filters.Filter.make_copy(self.resolve_option("setup")) self._filter.inputformat(data) filtered = self._filter.filter(data) if keep: filtered.relationname = relname self._output.append(Token(filtered)) else: if (self._filter is None) or self._header.equal_headers(data) is not None: self._header = Instances.template_instances(data) self._filter = filters.Filter.make_copy(self.resolve_option("setup")) self._filter.inputformat(data) filtered = self._filter.filter(data) if keep: filtered.relationname = relname self._output.append(Token(filtered.get_instance(0))) else: self._filter.input(inst) self._filter.batch_finished() filtered = self._filter.output() if keep: filtered.dataset.relationname = relname self._output.append(Token(filtered)) return None class DeleteFile(Transformer): """ Deletes the incoming files that match the regular expression. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(DeleteFile, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Deletes the incoming files that match the regular expression." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "regexp: " + str(self.config["regexp"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(DeleteFile, self).fix_config(options) opt = "regexp" if opt not in options: options[opt] = ".*" if opt not in self.help: self.help[opt] = "The regular expression that the files must match (string)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ fname = str(self.input.payload) spattern = str(self.resolve_option("regexp")) pattern = None if (spattern is not None) and (spattern != ".*"): pattern = re.compile(spattern) if (pattern is None) or (pattern.match(fname)): os.remove(fname) self._output.append(self.input) return None class CrossValidate(Transformer): """ Cross-validates the classifier/clusterer on the incoming dataset. In case of a classifier, the Evaluation object is forwarded. For clusterers the loglikelihood. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(CrossValidate, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Cross-validates the classifier/clusterer on the incoming dataset. In case of a classifier, the "\ "Evaluation object is forwarded. For clusterers the loglikelihood." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "setup: " + base.to_commandline(self.config["setup"]) + ", folds: " + str(self.config["folds"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(CrossValidate, self).fix_config(options) opt = "setup" if opt not in options: options[opt] = Classifier(classname="weka.classifiers.rules.ZeroR") if opt not in self.help: self.help[opt] = "The classifier/clusterer to train (Classifier/Clusterer)." opt = "folds" if opt not in options: options[opt] = 10 if opt not in self.help: self.help[opt] = "The number of folds for CV (int)." opt = "seed" if opt not in options: options[opt] = 1 if opt not in self.help: self.help[opt] = "The seed value for randomizing the data (int)." opt = "discard_predictions" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Discard classifier predictions to save memory (bool)." opt = "output" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "For capturing the classifier's prediction output (PredictionOutput)." return options def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if isinstance(token.payload, Instances): return raise Exception(self.full_name + ": Unhandled data type: " + str(token.payload.__class__.__name__)) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ data = self.input.payload cls = self.resolve_option("setup") if isinstance(cls, Classifier): cls = Classifier.make_copy(cls) evl = Evaluation(data) evl.discard_predictions = bool(self.resolve_option("discard_predictions")) evl.crossvalidate_model( cls, data, int(self.resolve_option("folds")), Random(int(self.resolve_option("seed"))), self.resolve_option("output")) self._output.append(Token(evl)) elif isinstance(cls, Clusterer): cls = Clusterer.make_copy(cls) evl = ClusterEvaluation() llh = evl.crossvalidate_model( cls, data, int(self.resolve_option("folds")), Random(int(self.resolve_option("seed")))) self._output.append(Token(llh)) else: return "Unhandled class: " + classes.get_classname(cls) return None class Evaluate(Transformer): """ Evaluates a trained classifier obtained from internal storage on the dataset passing through. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Evaluate, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Evaluates a trained classifier obtained from internal storage on the dataset passing through." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "storage: " + self.config["storage_name"] def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(Evaluate, self).fix_config(options) opt = "storage_name" if opt not in options: options[opt] = "unknown" if opt not in self.help: self.help[opt] = "The name of the classifier model in storage (string)." opt = "discard_predictions" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Discard classifier predictions to save memory (bool)." opt = "output" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "For capturing the classifier's prediction output (PredictionOutput)." return options def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if isinstance(token.payload, Instances): return raise Exception(self.full_name + ": Unhandled data type: " + str(token.payload.__class__.__name__)) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ data = self.input.payload if self.storagehandler is None: return "No storage handler available!" sname = str(self.resolve_option("storage_name")) if sname not in self.storagehandler.storage: return "No storage item called '" + sname + "' present!" cls = self.storagehandler.storage[sname] if isinstance(cls, Classifier): evl = Evaluation(data) evl.discard_predictions = bool(self.resolve_option("discard_predictions")) evl.test_model( cls, data, self.resolve_option("output")) elif isinstance(cls, Clusterer): evl = ClusterEvaluation() evl.set_model(cls) evl.test_model(data) else: return "Unhandled class: " + classes.get_classname(cls) self._output.append(Token(evl)) return None class EvaluationSummary(Transformer): """ Generates a summary string from an Evaluation/ClusterEvaluation object. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(EvaluationSummary, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Generates a summary string from an Evaluation object." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "title: " + str(self.config["title"]) \ + ", complexity: " + str(self.config["complexity"]) \ + ", matrix: " + str(self.config["matrix"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(EvaluationSummary, self).fix_config(options) opt = "title" if opt not in options: options[opt] = None if opt not in self.help: self.help[opt] = "The title for the output (string)." opt = "complexity" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to output classifier complexity information (bool)." opt = "matrix" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to output the classifier confusion matrix (bool)." return options def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if isinstance(token.payload, Evaluation): return None if isinstance(token.payload, ClusterEvaluation): return None raise Exception( self.full_name + ": Input token is not a supported Evaluation object - " + classes.get_classname(token.payload)) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ evl = self.input.payload if isinstance(evl, Evaluation): summary = evl.summary(title=self.resolve_option("title"), complexity=bool(self.resolve_option("complexity"))) if bool(self.resolve_option("matrix")): summary += "\n" + evl.matrix(title=self.resolve_option("title")) else: summary = evl.cluster_results self._output.append(Token(summary)) return None class ModelReader(Transformer): """ Reads the serialized model (Classifier/Clusterer) from disk and forwards a ModelContainer. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ModelReader, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Reads the serialized model from disk and forwards a ModelContainer." def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ fname = self.input.payload data = serialization.read_all(fname) if len(data) == 1: if javabridge.is_instance_of(data[0], "weka/classifiers/Classifier"): cont = ModelContainer(model=Classifier(jobject=data[0])) elif javabridge.is_instance_of(data[0], "weka/clusterers/Clusterer"): cont = ModelContainer(model=Clusterer(jobject=data[0])) else: return "Unhandled class: " + classes.get_classname(data[0]) elif len(data) == 2: if javabridge.is_instance_of(data[0], "weka/classifiers/Classifier"): cont = ModelContainer(model=Classifier(jobject=data[0]), header=Instances(data[1])) elif javabridge.is_instance_of(data[0], "weka/clusterers/Clusterer"): cont = ModelContainer(model=Clusterer(jobject=data[0]), header=Instances(data[1])) else: return "Unhandled class: " + classes.get_classname(data[0]) else: return "Expected 1 or 2 objects, but got " + str(len(data)) + " instead reading: " + fname self._output.append(Token(cont)) return None class Convert(Transformer): """ Converts the input data with the given conversion setup. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Convert, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Converts the input data with the given conversion setup." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "setup: " + str(self.config["setup"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ opt = "setup" if opt not in options: options[opt] = conversion.PassThrough() if opt not in self.help: self.help[opt] = "The conversion to apply to the input data (Conversion)." return super(Convert, self).fix_config(options) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if token is None: raise Exception(self.full_name + ": No token provided!") self.config["setup"].check_input(token.payload) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ conv = self.config["setup"].shallow_copy() conv.input = self._input.payload result = conv.convert() if result is None: if conv.output is not None: self._output.append(Token(conv.output)) return None class AttributeSelection(Transformer): """ Performs attribute selection on the incoming dataset using the specified search and evaluation scheme. Outputs a AttributeSelectionContainer with the results string, reduced dataset and seleted attributes. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(AttributeSelection, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return \ "Performs attribute selection on the incoming dataset using the specified search and evaluation scheme.\n" \ "Outputs a AttributeSelectionContainer with the results string, reduced dataset and seleted attributes." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "search: " + base.to_commandline(self.config["search"]) + ", eval: " \ + base.to_commandline(self.config["eval"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ opt = "search" if opt not in options: options[opt] = attsel.ASSearch(classname="weka.attributeSelection.BestFirst") if opt not in self.help: self.help[opt] = "The search algorithm to use (ASSearch)." opt = "eval" if opt not in options: options[opt] = attsel.ASEvaluation(classname="weka.attributeSelection.CfsSubsetEval") if opt not in self.help: self.help[opt] = "The evaluation algorithm to use (ASEvaluation)." return super(AttributeSelection, self).fix_config(options) def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if token is None: raise Exception(self.full_name + ": No token provided!") if not isinstance(token.payload, Instances): raise Exception(self.full_name + ": Not an Instances object!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ data = self.input.payload search = self.config["search"].shallow_copy() evl = self.config["eval"].shallow_copy() asel = attsel.AttributeSelection() asel.search(search) asel.evaluator(evl) asel.select_attributes(data) cont = AttributeSelectionContainer( original=data, reduced=asel.reduce_dimensionality(data), num_atts=asel.number_attributes_selected, selected=asel.selected_attributes, results=asel.results_string) self._output.append(Token(cont)) return None class RenameRelation(Transformer): """ Updates the relation name of Instance/Instances objects passing through. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(RenameRelation, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Updates the relation name of Instance/Instances objects passing through." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "name: " + str(self.config["name"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(RenameRelation, self).fix_config(options) opt = "name" if opt not in options: options[opt] = "newname" if opt not in self.help: self.help[opt] = "The new relation name to use (string)." return options def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if isinstance(token.payload, Instance): return if isinstance(token.payload, Instances): return raise Exception(self.full_name + ": Input token is neither an Instance nor Instances object!") def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ relname = self.resolve_option("name") if isinstance(self.input.payload, Instance): self.input.payload.dataset.relationname = relname else: self.input.payload.relationname = relname self._output.append(self.input) return None class Copy(Transformer): """ Creates a deep copy of the token passing through (must be a serializable JavaObject), other objects just get passed on. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Copy, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Creates a deep copy of the token passing through (must be a serializable JavaObject), "\ "other objects just get passed on." def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ if isinstance(self.input.payload, classes.JavaObject) and self.input.payload.is_serializable: copy = serialization.deepcopy(self.input.payload) if copy is not None: self._output.append(Token(copy)) else: self._output.append(self.input) return None class Predict(Transformer): """ Uses the serialized model or, if pointing to a directory, the specified model from storage for making a prediction on the incoming Instance object. The model can be either a Classifier or Clusterer. Outputs either a ClassificationContainer or ClusteringContainer. """ def __init__(self, name=None, config=None): """ Initializes the transformer. :param name: the name of the transformer :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Predict, self).__init__(name=name, config=config) self._model = None self._is_classifier = None def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return \ "Uses the serialized model or, if pointing to a directory, the specified model from storage for "\ "making a prediction on the incoming Instance object. The model can be either a Classifier or Clusterer.\n"\ "Outputs either a ClassificationContainer or ClusteringContainer." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "model: " + self.config["model"] + ", storage: " + self.config["storage_name"] def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(Predict, self).fix_config(options) opt = "model" if opt not in options: options[opt] = "." if opt not in self.help: self.help[opt] = "The serialized model to use for making predictions (string)." opt = "storage_name" if opt not in options: options[opt] = "unknown" if opt not in self.help: self.help[opt] = "The name of the model (or ModelContainer) in storage to use (string)." return options def check_input(self, token): """ Performs checks on the input token. Raises an exception if unsupported. :param token: the token to check :type token: Token """ if isinstance(token.payload, Instance): return raise Exception(self.full_name + ": Unhandled data type: " + str(token.payload.__class__.__name__)) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ inst = self.input.payload if not inst.has_class: return "No class set!" # load model? if self._model is None: model = None fname = str(self.resolve_option("model")) if os.path.isfile(fname): model = serialization.read(fname) else: name = self.resolve_option("storage_name") if name in self.storagehandler.storage: model = self.storagehandler.storage.get(name) if isinstance(model, ModelContainer): model = model.get("Model").jobject if model is None: return "No model available from storage or serialized file!" self._is_classifier = javabridge.is_instance_of(model, "weka/classifiers/Classifier") if self._is_classifier: self._model = Classifier(jobject=model) else: self._model = Clusterer(jobject=model) if self._is_classifier: cls = self._model.classify_instance(inst) dist = self._model.distribution_for_instance(inst) label = inst.class_attribute.value(int(cls)) cont = ClassificationContainer(inst=inst, classification=cls, distribution=dist, label=label) else: cls = self._model.cluster_instance(inst) dist = self._model.distribution_for_instance(inst) cont = ClusteringContainer(inst=inst, cluster=int(cls), distribution=dist) self._output.append(Token(cont)) return None python-weka-wrapper3-0.1.3/python/weka/flow/__init__.py0000644017267001726700000000144412712225123024502 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # __init__.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) # Simple workflow components, inspired by ADAMS (https://adams.cms.waikato.ac.nz/)python-weka-wrapper3-0.1.3/python/weka/flow/control.py0000644017267001726700000010756412712225142024436 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # control.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) import weka.flow.base as base from weka.flow.base import Actor, InputConsumer, OutputProducer, Stoppable, StorageHandler, Token from weka.flow.transformer import Transformer import weka.core.classes as classes class ActorHandler(Actor): """ The ancestor for all actors that handle other actors. """ def __init__(self, name=None, config=None): """ Initializes the actor handler. :param name: the name of the actor handler :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ActorHandler, self).__init__(name=name, config=config) self._director = self.new_director() if not classes.has_dict_handler("ActorHandler"): classes.register_dict_handler("ActorHandler", ActorHandler.from_dict) def new_director(self): """ Creates the director to use for handling the sub-actors. :return: the director instance :rtype: Director """ raise Exception("Not implemented!") def default_actors(self): """ Returns the default actors to use. :return: the default actors, if any :rtype: list """ return [] def check_actors(self, actors): """ Performs checks on the actors that are to be used. Raises an exception if invalid setup. :param actors: the actors to check :type actors: list """ pass def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(ActorHandler, self).fix_config(options) opt = "actors" if opt not in options: options[opt] = self.default_actors() if opt not in self.help: self.help[opt] = "The list of sub-actors that this actor manages." return options def to_dict(self): """ Returns a dictionary that represents this object, to be used for JSONification. :return: the object dictionary :rtype: dict """ result = super(ActorHandler, self).to_dict() result["type"] = "ActorHandler" del result["config"]["actors"] result["actors"] = [] for actor in self.actors: result["actors"].append(actor.to_dict()) return result @classmethod def from_dict(cls, d): """ Restores an object state from a dictionary, used in de-JSONification. :param d: the object dictionary :type d: dict :return: the object :rtype: object """ result = super(ActorHandler, cls).from_dict(d) if "actors" in d: l = d["actors"] for e in l: if u"type" in e: typestr = e[u"type"] else: typestr = e["type"] result.actors.append(classes.get_dict_handler(typestr)(e)) return result @property def actors(self): """ Obtains the currently set sub-actors. :return: the sub-actors :rtype: list """ result = self.config["actors"] if result is None: result = [] return result @actors.setter def actors(self, actors): """ Sets the sub-actors of the actor. :param actors: the sub-actors :type actors: list """ if actors is None: actors = self.default_actors() self.check_actors(actors) self.config["actors"] = actors @property def active(self): """ Returns the count of non-skipped actors. :return: the count :rtype: int """ result = 0 for actor in self.actors: if not actor.skip: result += 1 return result @property def first_active(self): """ Returns the first non-skipped actor. :return: the first active actor, None if not available :rtype: Actor """ result = None for actor in self.actors: if not actor.skip: result = actor break return result @property def last_active(self): """ Returns the last non-skipped actor. :return: the last active actor, None if not available :rtype: Actor """ result = None for actor in reversed(self.actors): if not actor.skip: result = actor break return result def index_of(self, name): """ Returns the index of the actor with the given name. :param name: the name of the Actor to find :type name: str :return: the index, -1 if not found :rtype: int """ result = -1 for index, actor in enumerate(self.actors): if actor.name == name: result = index break return result def update_parent(self): """ Updates the parent in its sub-actors. """ for actor in self.actors: actor.parent = self def setup(self): """ Configures the actor before execution. :return: None if successful, otherwise error message :rtype: str """ result = super(ActorHandler, self).setup() if result is None: self.update_parent() try: self.check_actors(self.actors) except Exception as e: result = str(e) if result is None: for actor in self.actors: name = actor.name newname = actor.unique_name(actor.name) if name != newname: actor.name = newname if result is None: for actor in self.actors: if actor.skip: continue result = actor.setup() if result is not None: break if result is None: result = self._director.setup() return result def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ return self._director.execute() def stop_execution(self): """ Triggers the stopping of the actor. """ self._director.stop_execution() def wrapup(self): """ Finishes up after execution finishes, does not remove any graphical output. """ for actor in self.actors: if actor.skip: continue actor.wrapup() super(ActorHandler, self).wrapup() def cleanup(self): """ Destructive finishing up after execution stopped. """ for actor in self.actors: if actor.skip: continue actor.cleanup() super(ActorHandler, self).cleanup() def _build_tree(self, actor, content): """ Builds the tree for the given actor. :param actor: the actor to process :type actor: Actor :param content: the rows of the tree collected so far :type content: list """ depth = actor.depth row = "" for i in range(depth - 1): row += "| " if depth > 0: row += "|-" name = actor.name if name != actor.__class__.__name__: name = actor.__class__.__name__ + " '" + name + "'" row += name quickinfo = actor.quickinfo if quickinfo is not None: row += " [" + quickinfo + "]" content.append(row) if isinstance(actor, ActorHandler): for sub in actor.actors: self._build_tree(sub, content) @property def tree(self): """ Returns a tree representation of this sub-flow. :return: the tree :rtype: str """ content = [] self._build_tree(self, content) return '\n'.join(content) class Director(object): """ Ancestor for classes that "direct" the flow of tokens. """ def __init__(self, owner): """ Initializes the director. :param owner: the owning actor :type owner: Actor """ self.check_owner(owner) self._owner = owner def __str__(self): """ Returns a short description about the director. :return: the description :rtype: str """ result = self.__name__ if self.owner is not None: result = result + ", owner=" + self.owner.name return result @property def owner(self): """ Obtains the currently set owner. :return: the owner :rtype: Actor """ return self._owner def check_owner(self, owner): """ Checks the owner. Raises an exception if invalid. :param owner: the owner to check :type owner: Actor """ pass def setup(self): """ Performs some checks. :return: None if successful, otherwise error message. :rtype: str """ return None def do_execute(self): """ Actual execution of the director. :return: None if successful, otherwise error message :rtype: str """ raise Exception("Not implemented!") def execute(self): """ Executes the director. :return: None if successful, otherwise error message :rtype: str """ if self.owner is None: return "No actor set as owner!" if self.owner.skip: return None return self.do_execute() class SequentialDirector(Director, Stoppable): """ Director for sequential execution of actors. """ def __init__(self, owner): """ Initializes the director. :param owner: the owning actor :type owner: Actor """ super(SequentialDirector, self).__init__(owner) self._stopping = False self._stopped = False self._allow_source = False self._record_output = True self._recorded_output = [] @property def allow_source(self): """ Obtains whether to allow a source actor. :return: true if to allow :rtype: bool """ return self._allow_source @allow_source.setter def allow_source(self, allow): """ Sets whether to allow a source actor. :param allow: true if to allow :type allow: bool """ self._allow_source = allow @property def record_output(self): """ Obtains whether to record the output of the last actor. :return: true if to record :rtype: bool """ return self._record_output @record_output.setter def record_output(self, record): """ Sets whether to record the output of the last actor. :param record: true if to record :type record: bool """ self._record_output = record @property def recorded_output(self): """ Obtains the recorded output. :return: the output :rtype: list """ return self._recorded_output def stop_execution(self): """ Triggers the stopping of the object. """ if not (self._stopping or self._stopped): for actor in self.owner.actors: actor.stop_execution() self._stopping = True def is_stopping(self): """ Returns whether the director is in the process of stopping. :return: """ return self._stopping def is_stopped(self): """ Returns whether the object has been stopped. :return: whether stopped :rtype: bool """ return self._stopped def check_owner(self, owner): """ Checks the owner. Raises an exception if invalid. :param owner: the owner to check :type owner: Actor """ if not isinstance(owner, ActorHandler): raise Exception("Owner is not an ActorHandler: " + owner.__name__) def check_actors(self): """ Checks the actors of the owner. Raises an exception if invalid. """ actors = [] for actor in self.owner.actors: if actor.skip: continue actors.append(actor) if len(actors) == 0: return if not self.allow_source and base.is_source(actors[0]): raise Exception("Actor '" + actors[0].full_name + "' is a source, but no sources allowed!") for i in range(1, len(actors)): if not isinstance(actors[i], InputConsumer): raise Exception("Actor does not accept any input: " + actors[i].full_name) def setup(self): """ Performs some checks. :return: None if successful, otherwise error message. :rtype: str """ result = super(SequentialDirector, self).setup() if result is None: try: self.check_actors() except Exception as e: result = str(e) return result def do_execute(self): """ Actual execution of the director. :return: None if successful, otherwise error message :rtype: str """ self._stopped = False self._stopping = False not_finished_actor = self.owner.first_active pending_actors = [] finished = False actor_result = None while not (self.is_stopping() or self.is_stopped()) and not finished: # determing starting point of next iteration if len(pending_actors) > 0: start_index = self.owner.index_of(pending_actors[-1].name) else: start_index = self.owner.index_of(not_finished_actor.name) not_finished_actor = None # iterate over actors token = None last_active = -1 if self.owner.active > 0: last_active = self.owner.last_active.index for i in range(start_index, last_active + 1): # do we have to stop the execution? if self.is_stopped() or self.is_stopping(): break curr = self.owner.actors[i] if curr.skip: continue # no token? get pending one or produce new one if token is None: if isinstance(curr, OutputProducer) and curr.has_output(): pending_actors.pop() else: actor_result = curr.execute() if actor_result is not None: self.owner.logger.error( curr.full_name + " generated following error output:\n" + actor_result) break if isinstance(curr, OutputProducer) and curr.has_output(): token = curr.output() else: token = None # still more to come? if isinstance(curr, OutputProducer) and curr.has_output(): pending_actors.append(curr) else: # process token curr.input = token actor_result = curr.execute() if actor_result is not None: self.owner.logger.error( curr.full_name + " generated following error output:\n" + actor_result) break # was a new token produced? if isinstance(curr, OutputProducer): if curr.has_output(): token = curr.output() else: token = None # still more to come? if curr.has_output(): pending_actors.append(curr) else: token = None # token from last actor generated? -> store if (i == self.owner.last_active.index) and (token is not None): if self._record_output: self._recorded_output.append(token) # no token produced, ignore rest of actors if isinstance(curr, OutputProducer) and (token is None): break # all actors finished? finished = (not_finished_actor is None) and (len(pending_actors) == 0) return actor_result class Flow(ActorHandler, StorageHandler): """ Root actor for defining and executing flows. """ def __init__(self, name=None, config=None): """ Initializes the sequence. :param name: the name of the sequence :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Flow, self).__init__(name=name, config=config) self._storage = {} def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Root actor for defining and executing flows." def new_director(self): """ Creates the director to use for handling the sub-actors. :return: the director instance :rtype: Director """ result = SequentialDirector(self) result.record_output = False result.allow_source = True return result def check_actors(self, actors): """ Performs checks on the actors that are to be used. Raises an exception if invalid setup. :param actors: the actors to check :type actors: list """ super(Flow, self).check_actors(actors) actor = self.first_active if (actor is not None) and not base.is_source(actor): raise Exception("First active actor is not a source: " + actor.full_name) @property def storage(self): """ Returns the internal storage. :return: the internal storage :rtype: dict """ return self._storage @classmethod def load(cls, fname): """ Loads the flow from a JSON file. :param fname: the file to load :type fname: str :return: the flow :rtype: Flow """ with open(fname) as f: content = f.readlines() return Flow.from_json(''.join(content)) @classmethod def save(cls, flow, fname): """ Saves the flow to a JSON file. :param flow: the flow to save :type flow: Flow :param fname: the file to load :type fname: str :return: None if successful, otherwise error message :rtype: str """ result = None try: f = open(fname, 'w') f.write(flow.to_json()) f.close() except Exception as e: result = str(e) return result class Sequence(ActorHandler, InputConsumer): """ Simple sequence of actors that get executed one after the other. Accepts input. """ def __init__(self, name=None, config=None): """ Initializes the sequence. :param name: the name of the sequence :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Sequence, self).__init__(name=name, config=config) self._output = [] def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Simple sequence of actors that get executed one after the other. Accepts input." def new_director(self): """ Creates the director to use for handling the sub-actors. :return: the director instance :rtype: Director """ result = SequentialDirector(self) result.record_output = False result.allow_source = False return result def check_actors(self, actors): """ Performs checks on the actors that are to be used. Raises an exception if invalid setup. :param actors: the actors to check :type actors: list """ super(Sequence, self).check_actors(actors) actor = self.first_active if (actor is not None) and not isinstance(actor, InputConsumer): raise Exception("First active actor does not accept input: " + actor.full_name) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ self.first_active.input = self.input result = self._director.execute() if result is None: self._output.append(self.input) return result class Tee(ActorHandler, Transformer): """ 'Tees off' the current token to be processed in the sub-tree before passing it on. """ def __init__(self, name=None, config=None): """ Initializes the sequence. :param name: the name of the sequence :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Tee, self).__init__(name=name, config=config) self._requires_active_actors = True def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "'Tees off' the current token to be processed in the sub-tree before passing it on." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ cond = str(self.config["condition"]) if len(cond) > 0: return "condition: " + cond else: return None def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(Tee, self).fix_config(options) opt = "condition" if opt not in options: options[opt] = "True" if opt not in self.help: self.help[opt] = "The (optional) condition for teeing off the tokens; uses the 'eval' method, "\ "ie the expression must evaluate to a boolean value; storage values placeholders "\ "'@{...}' get replaced with their string representations before evaluating the "\ "expression (string)." return options def new_director(self): """ Creates the director to use for handling the sub-actors. :return: the director instance :rtype: Director """ result = SequentialDirector(self) result.record_output = False result.allow_source = False return result def check_actors(self, actors): """ Performs checks on the actors that are to be used. Raises an exception if invalid setup. :param actors: the actors to check :type actors: list """ super(Tee, self).check_actors(actors) actor = self.first_active if actor is None: if self._requires_active_actors: raise Exception("No active actor!") elif not isinstance(actor, InputConsumer): raise Exception("First active actor does not accept input: " + actor.full_name) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None teeoff = True cond = self.storagehandler.expand(str(self.resolve_option("condition"))) if len(cond) > 0: teeoff = bool(eval(cond)) if teeoff: self.first_active.input = self.input result = self._director.execute() if result is None: self._output.append(self.input) return result class Trigger(ActorHandler, Transformer): """ 'Triggers' the sub-tree with each token passing through and then passes the token on. """ def __init__(self, name=None, config=None): """ Initializes the sequence. :param name: the name of the sequence :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Trigger, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "'Triggers' the sub-tree with each token passing through and then passes the token on." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ cond = str(self.resolve_option("condition")) if len(cond) > 0: return "condition: " + cond else: return None def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(Trigger, self).fix_config(options) opt = "condition" if opt not in options: options[opt] = "True" if opt not in self.help: self.help[opt] = "The (optional) condition for teeing off the tokens; uses the 'eval' method, "\ "ie the expression must evaluate to a boolean value; storage values placeholders "\ "'@{...}' get replaced with their string representations before evaluating the "\ "expression (string)." return options def new_director(self): """ Creates the director to use for handling the sub-actors. :return: the director instance :rtype: Director """ result = SequentialDirector(self) result.record_output = False result.allow_source = True return result def check_actors(self, actors): """ Performs checks on the actors that are to be used. Raises an exception if invalid setup. :param actors: the actors to check :type actors: list """ super(Trigger, self).check_actors(actors) actor = self.first_active if actor is None: raise Exception("No active actor!") if not base.is_source(actor): raise Exception("First active actor is not a source: " + actor.full_name) def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None teeoff = True cond = self.storagehandler.expand(str(self.resolve_option("condition"))) if len(cond) > 0: teeoff = bool(eval(cond)) if teeoff: result = self._director.execute() if result is None: self._output.append(self.input) return result class BranchDirector(Director, Stoppable): """ Director for the Branch actor. """ def __init__(self, owner): """ Initializes the director. :param owner: the owning actor :type owner: Actor """ super(BranchDirector, self).__init__(owner) self._stopping = False self._stopped = False def stop_execution(self): """ Triggers the stopping of the object. """ if not (self._stopping or self._stopped): for actor in self.owner.actors: actor.stop_execution() self._stopping = True def is_stopping(self): """ Returns whether the director is in the process of stopping. :return: """ return self._stopping def is_stopped(self): """ Returns whether the object has been stopped. :return: whether stopped :rtype: bool """ return self._stopped def check_owner(self, owner): """ Checks the owner. Raises an exception if invalid. :param owner: the owner to check :type owner: Actor """ if not isinstance(owner, Branch): raise Exception("Owner is not a Branch: " + owner.__name__) def check_actors(self): """ Checks the actors of the owner. Raises an exception if invalid. """ actors = [] for actor in self.owner.actors: if actor.skip: continue actors.append(actor) if len(actors) == 0: return for actor in actors: if not isinstance(actor, InputConsumer): raise Exception("Actor does not accept any input: " + actor.full_name) def setup(self): """ Performs some checks. :return: None if successful, otherwise error message. :rtype: str """ result = super(BranchDirector, self).setup() if result is None: try: self.check_actors() except Exception as e: result = str(e) return result def do_execute(self): """ Actual execution of the director. :return: None if successful, otherwise error message :rtype: str """ result = None self._stopped = False self._stopping = False for actor in self.owner.actors: if self.is_stopping() or self.is_stopped(): break actor.input = self.owner.input result = actor.execute() if result is not None: break return result class Branch(ActorHandler, InputConsumer): """ Passes on the input token to all of its sub-actors, one after the other. """ def __init__(self, name=None, config=None): """ Initializes the branch. :param name: the name of the branch :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Branch, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Passes on the input token to all of its sub-actors, one after the other." def new_director(self): """ Creates the director to use for handling the sub-actors. :return: the director instance :rtype: Director """ result = BranchDirector(self) return result class ContainerValuePicker(Tee): """ Picks the specified value from the container and 'tees' it off. """ def __init__(self, name=None, config=None): """ Initializes the actor. :param name: the name of the actor :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(ContainerValuePicker, self).__init__(name=name, config=config) self._requires_active_actors = False def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Picks the specified value from the container and 'tees' it off." @property def quickinfo(self): """ Returns a short string describing some of the options of the actor. :return: the info, None if not available :rtype: str """ return "value: " + str(self.config["value"]) + ", switch: " + str(self.config["switch"]) def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(ContainerValuePicker, self).fix_config(options) opt = "value" if opt not in options: options[opt] = "Model" if opt not in self.help: self.help[opt] = "The name of the container value to pick from the container (string)." opt = "switch" if opt not in options: options[opt] = False if opt not in self.help: self.help[opt] = "Whether to switch the ouputs, i.e., forward the container to the sub-flow and the " \ + "container value to the following actor instead (bool)." return options def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ result = None cont = self.input.payload name = str(self.resolve_option("value")) value = cont.get(name) switch = bool(self.resolve_option("switch")) if switch: if self.first_active is not None: self.first_active.input = self.input result = self._director.execute() if result is None: self._output.append(Token(value)) else: if self.first_active is not None: self.first_active.input = Token(value) result = self._director.execute() if result is None: self._output.append(self.input) return result class Stop(InputConsumer): """ Stops the execution of the flow. """ def __init__(self, name=None, config=None): """ Initializes the branch. :param name: the name of the branch :type name: str :param config: the dictionary with the options (str -> object). :type config: dict """ super(Stop, self).__init__(name=name, config=config) super(InputConsumer, self).__init__(name=name, config=config) def description(self): """ Returns a description of the actor. :return: the description :rtype: str """ return "Stops the execution of the flow." def do_execute(self): """ The actual execution of the actor. :return: None if successful, otherwise error message :rtype: str """ self.root.stop_execution() return None python-weka-wrapper3-0.1.3/python/weka/flow/conversion.py0000644017267001726700000001372412712225145025140 0ustar fracpetefracpete00000000000000# This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # conversion.py # Copyright (C) 2015-2016 Fracpete (pythonwekawrapper at gmail dot com) from weka.flow.base import Configurable import weka.core.classes as classes class Conversion(Configurable): """ Ancestor for conversions used by the 'Convert' transformer. """ def __init__(self, config=None): """ Initializes the conversion. :param config: dictionary of options to use :type config: dict """ super(Conversion, self).__init__(config=config) self._input = None self._output = None def __str__(self): """ Returns a short representation of the actor's setup. :return: the setup :rtype: str """ return classes.get_classname(self) + ": " + str(self._config) def check_input(self, obj): """ Performs checks on the input object. Raises an exception if unsupported. :param obj: the object to check :type obj: object """ pass @property def input(self): """ Returns the current input object, None if not available. :return: the input object :rtype: object """ return self._input @input.setter def input(self, obj): """ Accepts the data for processing. :param obj: the object to process :type obj: object """ self.check_input(obj) self._input = obj @property def output(self): """ Returns the generated output object, None if not available. :return: the output object :rtype: object """ return self._output def convert(self): """ Performs the actual conversion. :return: None if successful, otherwise errors message :rtype: str """ raise Exception("Not implemented!") class PassThrough(Conversion): """ Dummy conversion, just passes through the data. """ def __init__(self, config=None): """ Initializes the conversion. :param config: dictionary of options to use :type config: dict """ super(PassThrough, self).__init__(config=config) def description(self): """ Returns the description for the conversion. :return: the description :rtype: str """ return "Dummy conversion, just passes through the data." def convert(self): """ Performs the actual conversion. :return: None if successful, otherwise errors message :rtype: str """ self._output = self._input return None class AnyToCommandline(Conversion): """ Generates a commandline string, e.g., from a classifier. """ def __init__(self, config=None): """ Initializes the conversion. :param config: dictionary of options to use :type config: dict """ super(AnyToCommandline, self).__init__(config=config) def description(self): """ Returns the description for the conversion. :return: the description :rtype: str """ return "Generates a commandline string, e.g., from a classifier." def convert(self): """ Performs the actual conversion. :return: None if successful, otherwise errors message :rtype: str """ self._output = classes.to_commandline(self._input) return None class CommandlineToAny(Conversion): """ Generates an object from a commandline string, e.g., from a classifier setup. """ def __init__(self, config=None): """ Initializes the conversion. :param config: dictionary of options to use :type config: dict """ super(CommandlineToAny, self).__init__(config=config) def description(self): """ Returns the description for the conversion. :return: the description :rtype: str """ return "Generates an object from a commandline string, e.g., from a classifier setup." def fix_config(self, options): """ Fixes the options, if necessary. I.e., it adds all required elements to the dictionary. :param options: the options to fix :type options: dict :return: the (potentially) fixed options :rtype: dict """ options = super(CommandlineToAny, self).fix_config(options) opt = "wrapper" if opt not in options: options[opt] = "weka.core.classes.OptionHandler" if opt not in self.help: self.help[opt] = "The name of the wrapper class to use (string)." return options def check_input(self, obj): """ Performs checks on the input object. Raises an exception if unsupported. :param obj: the object to check :type obj: object """ if isinstance(obj, str): return if isinstance(obj, unicode): return raise Exception("Unsupported class: " + self._input.__class__.__name__) def convert(self): """ Performs the actual conversion. :return: None if successful, otherwise errors message :rtype: str """ cname = str(self.config["wrapper"]) self._output = classes.from_commandline(self._input, classname=cname) return None python-weka-wrapper3-0.1.3/setup.cfg0000644017267001726700000000023713147133421020774 0ustar fracpetefracpete00000000000000[build_sphinx] source-dir = doc/source build-dir = doc/build all_files = 1 [upload_sphinx] upload-dir = doc/build/html [egg_info] tag_build = tag_date = 0