PyBMF

A Python library for Boolean Matrix Factorization. Work under Preferred.ai.

PyBMF is under active development. We welcome the authors of BMF papers and those interested in BMF to play around and contribute. Please contact us if you have any questions or suggestions.

Prospectives

Boolean matrix factorization (BMF) is a well-known problem in pattern mining. Throughout the years of prosperous research, it has evolved from greedy heuristics to include a wide range of advanced technologies. We hold the belief that a playground with fairness and adaptiveness is necessary for the development of such algorithms.

PyBMF aims to provide a unified framework with:

1. generators for various types of synthetic data

2. easy ways of importing and sampling real-world datasets like MovieLensData and NetflixData

3. data RatioSplit and CrossValidation utilities

4. tools for generating negative_sample() when needed

5. compatibility of scipy.sparse matrices when it can

6. tools to evaluate() using binary and continuous metrics

7. visualization tools to show_matrix() in single or multi-matrix mode

8. tools to save_model and show_logs in HTML or OverLeaf with logs2html and logs2latex

9. ability to incorporate Boolean matrix simplification and visualization models in planned future

Models

Category	Model	Paper	Original Implementation	In PyBMF
Heuristics	Asso	PKDD2006 TKDE2008	C	✅
Heuristics	Hyper/Hyper+	SIGKDD2011		✅
Heuristics	GreConD	JCSS2010	MATLAB	✅
Heuristics	Panda	ICDM2010		✅
Heuristics	Panda+	TKDE2013		✅
Heuristics	NASSAU	SDM2015	link
Heuristics	GreConD+	DAM2018	MATLAB	✅
Heuristics	MEBF	AAAI2020	R	✅
Continuous	NMFSklearn			🛞 Wrapper of sklearn.nmf
Continuous	WNMF			✅ Multiplicative update
Continuous	BinaryMF-Penalty	ICDM2007	MATLAB	✅ Multiplicative update
Continuous	BinaryMF-Thresholding	ICDM2007	MATLAB	✅ Line search
Continuous	FastStep	PAKDD2016	C++	✅ Line search
Continuous	PRIMP	DMKD2017	CUDA C++	✅ PALM
Continuous	PNL-PF	SP2021		✅ Multiplicative update
Continuous	ELBMF	NIPS2022	Julia Python	✅ PALM
Probablistic	MessagePassing	ICML2016	Python	🛞 Wrapper of original implementation
Probablistic	OrMachine	ICLM2017	Cython	🛞 Wrapper of original implementation
Linear Optimization	ColumnGeneration	AAAI2021	Python	🛞 Wrapper of original implementation
Satisfiability	UndercoverBMF	AAAI2021	C++	🛞 Wrapper of original implementation
Simplification	IterEss	IS2019
Simplification	DelegationBMF	AAAI2024	C++
Visualization	OrderedBMF	SIAM2019	C++
Visualization	BiclusterVisualization	PKDD2023	Python

How to use PyBMF

Check Examples that help you get started with PyBMF.

Check Models in which you can implement your own models.

Compatibility

Currently built and tested on Python 3.9.18.

TO-DO

• [x] Diagnosis of thresholding models

• [x] Fix DataFrame display utils in dataframe_utils.py

• [ ] Add mask parameter W to PRIMP and ELBMF

• [ ] Make a page dedicated to contributors and references

• [ ] Include BMF visualization models

• [ ] Include BMF simplification models

Subpackages

• PyBMF.datasets package
  • Submodules
  • PyBMF.datasets.BaseData module
    • BaseData
      • BaseData.dump_pickle()
      • BaseData.has_pickle
      • BaseData.load()
      • BaseData.load_data()
      • BaseData.read_data()
      • BaseData.read_pickle()
      • BaseData.sample()
      • BaseData.show_matrix()
      • BaseData.to_single()
  • PyBMF.datasets.BaseSplit module
    • BaseSplit
      • BaseSplit.check_params()
      • BaseSplit.get_neg_indices()
      • BaseSplit.load_neg_data()
      • BaseSplit.load_pos_data()
      • BaseSplit.negative_sample()
  • PyBMF.datasets.CrossValidation module
    • CrossValidation
      • CrossValidation.get_fold()
      • CrossValidation.get_indices()
      • CrossValidation.get_partition()
      • CrossValidation.negative_sample()
  • PyBMF.datasets.MovieLensData module
    • MovieLensData
      • MovieLensData.load_data()
      • MovieLensData.read_data()
      • MovieLensData.sort_factor()
  • PyBMF.datasets.MovieLensGenreCastData module
    • MovieLensGenreCastData
      • MovieLensGenreCastData.get_attribute_info()
      • MovieLensGenreCastData.load_data()
      • MovieLensGenreCastData.read_data()
  • PyBMF.datasets.MovieLensGenreCastUserData module
    • MovieLensGenreCastUserData
      • MovieLensGenreCastUserData.load_data()
      • MovieLensGenreCastUserData.read_data()
  • PyBMF.datasets.MovieLensUserData module
    • MovieLensUserData
      • MovieLensUserData.get_user_profile()
      • MovieLensUserData.load_data()
      • MovieLensUserData.read_data()
  • PyBMF.datasets.NetflixData module
    • NetflixData
      • NetflixData.load_data()
      • NetflixData.read_data()
  • PyBMF.datasets.NetflixGenreCastData module
    • NetflixGenreCastData
      • NetflixGenreCastData.get_attribute_info()
      • NetflixGenreCastData.load_data()
      • NetflixGenreCastData.read_data()
  • PyBMF.datasets.NoSplit module
    • NoSplit
      • NoSplit.negative_sample()
  • PyBMF.datasets.RatioSplit module
    • RatioSplit
      • RatioSplit.get_indices()
      • RatioSplit.get_size()
      • RatioSplit.negative_sample()
  • Module contents
    • CrossValidation
      • CrossValidation.get_fold()
      • CrossValidation.get_indices()
      • CrossValidation.get_partition()
      • CrossValidation.negative_sample()
    • MovieLensData
      • MovieLensData.load_data()
      • MovieLensData.read_data()
      • MovieLensData.sort_factor()
    • MovieLensGenreCastData
      • MovieLensGenreCastData.get_attribute_info()
      • MovieLensGenreCastData.load_data()
      • MovieLensGenreCastData.read_data()
    • MovieLensGenreCastUserData
      • MovieLensGenreCastUserData.load_data()
      • MovieLensGenreCastUserData.read_data()
    • MovieLensUserData
      • MovieLensUserData.get_user_profile()
      • MovieLensUserData.load_data()
      • MovieLensUserData.read_data()
    • NetflixData
      • NetflixData.load_data()
      • NetflixData.read_data()
    • NetflixGenreCastData
      • NetflixGenreCastData.get_attribute_info()
      • NetflixGenreCastData.load_data()
      • NetflixGenreCastData.read_data()
    • NoSplit
      • NoSplit.negative_sample()
    • RatioSplit
      • RatioSplit.get_indices()
      • RatioSplit.get_size()
      • RatioSplit.negative_sample()
• PyBMF.generators package
  • Submodules
  • PyBMF.generators.BaseGenerator module
    • BaseGenerator
      • BaseGenerator.add_noise()
      • BaseGenerator.boolean_matmul()
      • BaseGenerator.check_params()
      • BaseGenerator.generate()
      • BaseGenerator.generate_factor()
      • BaseGenerator.generate_factors()
      • BaseGenerator.measure()
      • BaseGenerator.measure_density()
      • BaseGenerator.measure_overlap()
      • BaseGenerator.set_config()
      • BaseGenerator.set_factor_info()
      • BaseGenerator.set_params()
      • BaseGenerator.show_matrix()
      • BaseGenerator.shuffle()
      • BaseGenerator.shuffle_factors()
      • BaseGenerator.sorted_index()
      • BaseGenerator.sortout()
      • BaseGenerator.to_dense()
      • BaseGenerator.to_sparse()
  • PyBMF.generators.BlockDiagonalMatrixGenerator module
    • BlockDiagonalMatrixGenerator
      • BlockDiagonalMatrixGenerator.generate()
      • BlockDiagonalMatrixGenerator.generate_factor()
      • BlockDiagonalMatrixGenerator.generate_factors()
  • PyBMF.generators.BlockMatrixGenerator module
    • BlockMatrixGenerator
      • BlockMatrixGenerator.check_overlap()
      • BlockMatrixGenerator.generate()
      • BlockMatrixGenerator.generate_factor()
      • BlockMatrixGenerator.generate_factor_points()
      • BlockMatrixGenerator.generate_factors()
      • BlockMatrixGenerator.is_overlapped()
  • PyBMF.generators.SyntheticMatrixGenerator module
    • SyntheticMatrixGenerator
      • SyntheticMatrixGenerator.generate()
      • SyntheticMatrixGenerator.generate_factor()
      • SyntheticMatrixGenerator.generate_factors()
  • Module contents
    • BlockDiagonalMatrixGenerator
      • BlockDiagonalMatrixGenerator.generate()
      • BlockDiagonalMatrixGenerator.generate_factor()
      • BlockDiagonalMatrixGenerator.generate_factors()
• PyBMF.models package
  • Submodules
  • PyBMF.models.Asso module
    • Asso
      • Asso._fit()
      • Asso.fit()
      • Asso.init_model()
    • build_assoc()
    • build_basis()
    • get_vector()
  • PyBMF.models.AssoIter module
    • AssoIter
      • AssoIter._fit()
      • AssoIter.check_params()
      • AssoIter.fit()
      • AssoIter.get_refined_column()
      • AssoIter.init_model()
  • PyBMF.models.AssoOpt module
    • AssoOpt
      • AssoOpt._fit()
      • AssoOpt.check_params()
      • AssoOpt.fit()
      • AssoOpt.init_model()
      • AssoOpt.set_optimal_row()
    • int2bin()
  • PyBMF.models.BaseModel module
    • BaseModel
      • BaseModel._evaluate()
      • BaseModel._fit()
      • BaseModel.check_params()
      • BaseModel.evaluate()
      • BaseModel.finish()
      • BaseModel.fit()
      • BaseModel.init_model()
      • BaseModel.load_dataset()
      • BaseModel.predict_X()
      • BaseModel.show_matrix()
  • PyBMF.models.BaseModelTools module
    • BaseModelTools
      • BaseModelTools._early_stop()
      • BaseModelTools._init_factors()
      • BaseModelTools._init_logs()
      • BaseModelTools._make_name()
      • BaseModelTools._save_model()
      • BaseModelTools._show_logs()
      • BaseModelTools._show_result()
      • BaseModelTools._start_timer()
      • BaseModelTools._stop_timer()
      • BaseModelTools.early_stop()
      • BaseModelTools.extend_factors()
      • BaseModelTools.import_model()
      • BaseModelTools.print_msg()
      • BaseModelTools.set_config()
      • BaseModelTools.set_factors()
      • BaseModelTools.set_params()
      • BaseModelTools.truncate_factors()
  • PyBMF.models.BinaryMFPenalty module
    • BinaryMFPenalty
      • BinaryMFPenalty._fit()
      • BinaryMFPenalty.check_params()
      • BinaryMFPenalty.fit()
      • BinaryMFPenalty.get_prediction()
      • BinaryMFPenalty.update_U()
      • BinaryMFPenalty.update_V()
    • error()
    • rec_error()
    • reg_error()
    • update_U()
    • update_V()
  • PyBMF.models.BinaryMFThreshold module
    • BinaryMFThreshold
      • BinaryMFThreshold.F()
      • BinaryMFThreshold._fit()
      • BinaryMFThreshold.check_params()
      • BinaryMFThreshold.dF()
      • BinaryMFThreshold.dXdx()
      • BinaryMFThreshold.evaluate_with_threshold()
      • BinaryMFThreshold.fit()
      • BinaryMFThreshold.threshold_to_x()
      • BinaryMFThreshold.x_bounds()
      • BinaryMFThreshold.x_to_threshold()
  • PyBMF.models.ContinuousModel module
    • ContinuousModel
      • ContinuousModel._show_matrix()
      • ContinuousModel._to_bool()
      • ContinuousModel._to_dense()
      • ContinuousModel._to_float()
      • ContinuousModel.init_UV()
      • ContinuousModel.init_W()
      • ContinuousModel.init_model()
      • ContinuousModel.normalize_UV()
      • ContinuousModel.show_matrix()
    • unique_values_mapping()
  • PyBMF.models.ELBMF module
    • ELBMF
      • ELBMF.fit()
      • ELBMF.iPALM()
      • ELBMF.init_model()
    • get_integrality_gap()
    • prox()
    • update_U()
  • PyBMF.models.ELBMFNumPy module
    • ELBMF
      • ELBMF.factorize()
      • ELBMF.init_factorization()
      • ELBMF.print_loss()
  • PyBMF.models.ELBMFPyTorch module
  • PyBMF.models.FastStep module
    • FastStep
      • FastStep.F()
      • FastStep._fit()
      • FastStep.check_params()
      • FastStep.dF()
      • FastStep.early_stop()
      • FastStep.fit()
  • PyBMF.models.GreConD module
    • GreConD
      • GreConD._fit()
      • GreConD.fit()
    • get_concept()
  • PyBMF.models.GreConDPlus module
    • GreConDPlus
      • GreConDPlus.fit()
      • GreConDPlus.remove_covered()
      • GreConDPlus.remove_overlapped()
      • GreConDPlus.set_extensions()
    • _expansion()
    • expansion()
  • PyBMF.models.Hyper module
    • Hyper
      • Hyper.cost()
      • Hyper.find_hyper()
      • Hyper.fit()
      • Hyper.init_itemsets()
      • Hyper.init_model()
      • Hyper.init_transactions()
      • Hyper.sort_by_cost()
  • PyBMF.models.HyperPlus module
    • HyperPlus
      • HyperPlus.check_params()
      • HyperPlus.fit()
    • cost_savings()
  • PyBMF.models.IP module
    • IP
  • PyBMF.models.MEBF module
    • MEBF
      • MEBF._fit()
      • MEBF.bidirectional_growth()
      • MEBF.fit()
      • MEBF.get_factor()
      • MEBF.get_weak_signal()
      • MEBF.weak_signal_detection()
  • PyBMF.models.MaxSAT module
    • MaxSAT
      • MaxSAT.fit()
  • PyBMF.models.MessagePassing module
  • PyBMF.models.NMFSklearn module
    • NMFSklearn
      • NMFSklearn.check_params()
      • NMFSklearn.fit()
  • PyBMF.models.OrMachine module
    • OrMachine
      • OrMachine.fit()
  • PyBMF.models.PNLPF module
    • PNLPF
      • PNLPF.get_prediction()
      • PNLPF.update_U()
      • PNLPF.update_V()
    • get_prediction_with_sigmoid()
    • update_U()
    • update_V()
  • PyBMF.models.PRIMP module
  • PyBMF.models.PRIMPPyTorch module
  • PyBMF.models.Panda module
    • Panda
      • Panda._fit()
      • Panda.check_params()
      • Panda.extend_core()
      • Panda.find_core()
      • Panda.fit()
      • Panda.sort_items()
  • PyBMF.models.TransposedModel module
    • TransposedModel
      • TransposedModel.fit()
  • PyBMF.models.WNMF module
    • WNMF
      • WNMF._fit()
      • WNMF.check_params()
      • WNMF.error()
      • WNMF.fit()
      • WNMF.update()
  • Module contents
• PyBMF.solvers package
  • Submodules
  • PyBMF.solvers.line_search module
    • limit_step_size()
    • line_search()
  • Module contents
    • line_search()
• PyBMF.utils package
  • Submodules
  • PyBMF.utils.boolean_utils module
    • add()
    • dot()
    • ismat()
    • isnum()
    • matmul()
    • multiply()
    • power()
    • subtract()
  • PyBMF.utils.collective_display_utils module
    • get_settings()
    • sort_matrices()
  • PyBMF.utils.collective_evaluate_utils module
    • collective_cover()
    • harmonic_score()
    • weighted_score()
  • PyBMF.utils.collective_transform_utils module
    • concat_Us_into_U()
    • concat_Xs_into_X()
    • concat_factor_info()
    • split_U_into_Us()
    • split_X_into_Xs()
  • PyBMF.utils.collective_utils module
    • get_dummy_factor_info()
    • get_factor_dims()
    • get_factor_list()
    • get_factor_starts()
    • get_matrices()
    • split_factor_list()
  • PyBMF.utils.common module
    • binarize()
    • d_sigmoid()
    • get_prediction()
    • get_prediction_with_threshold()
    • get_residual()
    • get_rng()
    • safe_indexing()
    • to_interval()
  • PyBMF.utils.data_utils module
    • mean()
    • median()
    • sample()
    • sort_order()
    • sum()
    • summarize()
  • PyBMF.utils.dataframe_utils module
    • _make_html()
    • _make_name()
    • _open_html()
    • get_config()
    • log2html()
    • log2latex()
  • PyBMF.utils.decorator_utils module
    • ignore_warnings()
    • timeit()
  • PyBMF.utils.display module
    • fill_nan()
    • get_size_inches()
    • show_factor_distribution()
    • show_matrix()
  • PyBMF.utils.evaluate_utils module
    • eval()
    • header()
    • record()
  • PyBMF.utils.experiment_utils module
    • get_model_by_path()
    • get_model_by_time()
  • PyBMF.utils.generator_utils module
    • add_noise()
    • reverse_index()
    • shuffle_by_dim()
    • shuffle_matrix()
  • PyBMF.utils.metrics module
    • ACC()
    • ERR()
    • F1()
    • FN()
    • FNR()
    • FP()
    • FPR()
    • MAE()
    • PPV()
    • RMSE()
    • TN()
    • TNR()
    • TP()
    • TPR()
    • coverage_score()
    • description_length()
    • get_metrics()
    • invert()
    • weighted_error()
  • PyBMF.utils.sparse_utils module
    • bool_to_index()
    • check_sparse()
    • index_to_bool()
    • sparse_indexing()
    • to_dense()
    • to_sparse()
    • to_triplet()
  • Module contents

Indices and tables

• Index

• Module Index

• Search Page