python - Which C++ libraries should I use for a large parallel computing number-crunching project exploiting third-party applications -

introduction

i want request lot of advice on new programming project going start on own. going precise in accomplish , in basic requirements are. therefore going long question. please bear me.

i going split question 5 sections:

1. real-world problem
2. simulation problem
3. requirements , preferences
4. additional information
5. list of advice requests

1. real-world problem

skyscrapers , large bridges suffer dynamic wind loading. means, when designed incorrectly can collapse due wind-induced vibrations (this happened in 1940: http://www.youtube.com/watch?v=3mclp9qmcgs). design such structures correctly, efficient number-crunching software required analysis , simulation.

2. simulation problem

there exists multitude of software capable of either simulating fluid flows or structural mechanics. many have been developed on 30 years , proven , mature technologies. writing multi-physical program capable of simulating both fluid flows , structural mechanics simultaneously scratch, therefore unwise. first of all, need years of development before reaching maturity , hard enter world has depended on specific software on 30 years. more important...why recreate when can reuse? instead of pursuing monolithic approach, prefer partitioned approach can reuse existing simulation software.

in partitioned approach use software x simulate flows , use software y simulate structures. write own coupling algorithm establishes communication between x , y , uses them simulate multi-physical problem (e.g. wind-induced vibrations of skyscrapers or bridges). reason use x , y , not actual software names because x , y supposed black-boxes. in no way coupling algorithm dependent on implementation of x , y. algorithm depend on output of x , y. way end-user can select ever x or y available them or ever x or y capable of doing end-user wants achieve.

because want use black-box partitioned approach, software x knows nothing of y , vice-versa. how simulate deformation of bridge without knowing of surrounding air flow , how know in way surrounding air flow perturbed structure without knowing deformation? answer simple: start guess , use iterative approach converge correct solution. approach computationally expensive. reduce computational cost coupling algorithm can written in clever way using efficient technologies, not discussed here. heavy linear algebra number-crunching required.

3. requirements , preferences

what need is:

• establish communication between third-party open-source or proprietary software
• perform heavy number-crunching (linear algebra)
• visualise results (2d / 3d plotting , animating)
• deliver interactive analysis , development environment
• create intuitive graphical user interface

what want software be:

• open-source
• cross-platform
• extendable through scripts and/or shared libraries

what going use:

• c++ heavy number-crunching
• cpython programming logic
• numpy / scipy number-crunching in cpython
• matplotlib results visualisation in cpython

4. additional information

facts:

• one-man project @ start, grow company if successful
• primary os kde-based linux distribution

business model:

• free software , basic documentation.
• paid services , elaborate documentation.

5. list of advice requests

i want number-crunching in c++ writing many functions individually perform tiny task. program logic contained in cpython package executes entire simulation while relying on c++ functions perform number-crunching. c++ / cpython algorithm extended scripts written in cpython (using numpy, scipy, sympy , matplotlib) generate , visualise results raw numerical data. want able parallel computing , need communicate several third party open-source , proprietary software.

to accomplish going need whole bunch of existing libraries/packages/technologies etc. , relevant issues know can use, not know should use. best solution try out , see works best. if experienced user can weed out of more unlikely candidates gladly receive or hers advice, suggestion, pro / con list on:

1. glueing c++ , cpython (e.g. ctypes, sip, swig etc.)
2. c++ linear algebra number-crunching library (e.g. armadillo, eigen, petsc etc.)
3. graphical interface development library (e.g. qt, gtk, wxwidgets etc.)
4. software communication , parallel computing (e.g. mpich, openmpi, openmp etc.)
5. cpython 2.7.x or cpython 3.x

note: have summed options above, these exemplary , not limitation to. open long written in c, c++, fortran or python. not expect answer in 5 categories enlisted above 1 individual. let collective knowledge of community take care of that.

i thank contributors , wish best of luck in own endeavors.

you mention parallelism not how intend make project parallel. more complex issue choosing couple libraries. there several major considerations required before moving forward.

you mention intended platform briefly, have consider whether simulation run on single computer/node or multiple. considering doing iterative simulation of building, going require far more compute power 1 computer can provide. means that, unless want go hybrid multiple-process, multiple-thread approach, limited multiple-process model of parallelism. opencl , mpi each options implementation. (note: mpich , openmpi each implementations of mpi, , code should agnostic of these) message passing mpi general model of parallelism can quite difficult not used working parallel code. personal experience mpi , hybrid programming cannot else regarding choice of parallel model.

a problem follows issue of parallel model directly impacts simulation software. not entirely how separate planning on keeping algorithm simulations. if plan have code fork separate process run simulations, have issues cross-platform support may not have luxury of being able run arbitrary simulations in manner. if instead intend run simulations within software, parallel model has consistent throughout. although puts limitations on black box strategy, may make entire thing more feasible.

a deal has been said applicable libraries. don't have more specific libraries hasn't been said. keep in mind many of same issues have addressed these when running simulations.

tldr: parallelism should not looked over. need know parallel model using before making decisions on libraries.