\n\n\n\n
![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/cvt_new.png\")
<\/figure>Sampling and surrogates for simulations: <\/strong>Analysis of data from computer simulations presents unique opportunities for data analysis. Simulations of complex physical phenomena can be very expensive to run, but at the same time, we have greater control over the data we choose to generate. For the last fifteen years, I have been interested in how we can combine the generation and the analysis of the data to reduce the cost of gaining insight into the phenomenon being simulated. Two ideas play a role – sampling, where we carefully select the values of the input parameters at which to run the simulation, and surrogate models, which are fast, but approximate, alternatives to the simulation. By combining these ideas suitably, we can identify viable regions in the input space; solve inverse problems, where we seek the input parameters that result in specific output values with associated uncertainties; design experiments; and progressively refine our understanding of the phenomenon being simulated.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Sampling for surrogate models, hyperparameter optimization, and data analysis: <\/strong>The quality of a surrogate model depends not only on the model used, but also the sample points at which the training data are generated. Often, the focus is on the initial set of sample points used to create the model. But, as the practical use of these models increases, there is a need for algorithms that not only generate space-filling samples, but also support progressive and incremental sampling. We explore algorithms used in various disciplines and evaluate them in the context of how well they support the many needs of modern surrogate modeling, the closely related task of hyperparameter optimization, and data analysis in general.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Regression with small data sets: <\/strong>Surrogate modeling is often used to create a fast, but approximate, alternative to simulations. By running the simulation at a few carefully-chosen sample points in the input parameter space, we can use the corresponding inputs-outputs as a training data set to build a machine learning model that acts as a surrogate for the simulation. However, for expensive simulations, when we can generate only a small training set, it is unclear if some machine learning models perform better than others. We compared several popular models, evaluating them not just on prediction quality, but also on their applicability to practical problems, such as, identifying the viable region of a process, solving inverse problems, and identifying parameter values for use in experiments.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Interpreting the solution to inverse problems: <\/strong>In earlier work, we have combined sampling and code surrogates to solve inverse problems, where we want to find input parameters that map to target output values, often specified with associated uncertainties. However, interpreting the solution is a challenge, especially when the input space is high-dimensional. The solution is often difficult to visualize as it can span a large range of values in each input dimension, even though it occupies a small fraction of the total hyper-volume spanned by this range of values. We have explored the use of self-organizing maps to map the solution to a lower dimensional space so we can understand where the solution lies in the input space of the problem, enabling us to use the solution in practice.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Independent-block Gaussian process:<\/strong> Gaussian process is a popular model for regression, as it provides not just a prediction, but the uncertainty as well. However, it can be expensive, requiring the solution of a linear system of equations. We investigated the use of tapering, where small elements in the covariance matrix are dropped, and combined it with reordering schemes to create a banded covariance matrix for a faster solution. However, we found the idea does not work in general. Instead, motivated by the concept of block tapering, we proposed the independent-block GP method, which is a simple way to reduce the cost of solution without sacrificing the accuracy of the predictions. The method is also embarrassingly parallel, leading to further reduction in computational cost.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Compressing simulation data: <\/strong>Computer simulations can generate vast quantities of floating point data, making compression a key aspect of the I\/O and storage of the data. However, when the data are unstructured, it becomes a challenge to identify neighboring data points so we can exploit the similarity among them to aid in the compression. For both lossy and lossless compression of unstructured simulation data, we explored the use of compressive sensing, sampling combined with regression, and clustering techniques (collaboration with Prof. George Karypis, from the University of Minnesota).<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Intelligent exploration of large-scale data. <\/strong>A challenge in data analysis is the selection of algorithms, and their parameters, for use in each step of the analysis. These choices are often made by examining the data and through trial and error. When the data set is too large to permit easy visualization and exploration, we typically select a sample of data points and examine them to understand the characteristics of the data. We consider alternatives to a simple random selection of samples to understand how we can learn more about the data set, especially when we are restricted to a small number of passes through the data.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Analysis of time series data from sensors: <\/strong>Time series data from sensors can be analyzed to understand and gain insight into the quantities being measured. Using data mainly from wind-energy applications, we show how we can identify diurnal motifs or recurring patterns, predict imminent changes in the wind energy, and identify important sensor streams. These ideas could provide energy operators additional information they could exploit in scheduling wind energy on the power grid.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Dimension reduction for scientific applications:<\/strong> Reducing the number of dimensions, that is, the number of features representing a data point, is important in scientific applications to minimize the effect of irrelevant or redundant features in any subsequent analysis. Often many different types of features are extracted for each data point using a range of techniques, and domain information alone may not be sufficient to prune the features to keep only the relevant ones. We investigated filters, wrappers, and several non-linear dimension reduction techniques for their effectiveness in scientific applications ranging from remote sensing to astronomy and plasma physics.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n ASPEN – Approximate splitting for ensembles: <\/strong>Ensembles of classifiers, where different classifiers are created from the same data set through randomization, can improve the classification accuracy. To reduce the cost of creating multiple classifiers, we considered two ways to randomize the split decision at each node of the tree – use a sub-sample of instances at the node to identify the best split, or create a histogram, evaluate splits at the mid-point of each bin, and select the split randomly in the bin that contains the best split. A combination of both ideas can furthur reduce the cost of building the ensemble.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Tracking moving objects in simulations and video:<\/strong> Detection and tracking of moving objects are important tasks in problems such as activity detection and identification in video sequences. We explored a range of techniques, focusing on how to make them more robust and computationally efficient so we could detect and track a moderate number of vehicles in video from traffic sequences, as well as a large number of non-rigid, coherent structures in spatio-temporal data from simulations.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n PDEs for image processing<\/strong>: Partial differential equations have been used for image processing tasks such as denoising and segmentation. In some of our early work, we evaluated the performance of these methods on real images so we could understand better their pros and cons and compare them with more traditional methods of image processing. We were particularly interested in the computational cost of the PDE-based methods and the choice of various parameters and options in their implementation.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n Sapphire scientific data mining software<\/strong> (R&D100 award, 2006): When I started the Sapphire scientific data mining project in late 1990s, I put together an object-oriented, modular design for the software. A common interface for each class of algorithms allowed us to easily try different algorithms and create new ones, a common data store supported many different data formats used in different domains, and the implementation of the compute intensive parts in C++, that were glued together using Python, enabled us to quickly create efficient solutions to specific problems. Nearly twenty-five years later, with lots more practical experience in data mining, I find the approach I took provided both the flexibility and efficiency required to meet the diverse needs of data analysis in scientific simulations, observations and experiments.<\/p>\n<\/div><\/div>\n\n\n\n Select publications (available from Google Scholar<\/a>):<\/p>\n\n\n\n This page describes my research in data mining algorithms, that is, solution techniques for specific tasks in data analysis. Unlike the area of applications, where multiple challenges often have to be addressed simultaneously, a focus on the algorithms enables me to consider, in isolation, each of the many challenges encountered in real applications. I canContinue reading “Research in Algorithms”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":192,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/pages\/197"}],"collection":[{"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/comments?post=197"}],"version-history":[{"count":43,"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/pages\/197\/revisions"}],"predecessor-version":[{"id":525,"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/pages\/197\/revisions\/525"}],"up":[{"embeddable":true,"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/pages\/192"}],"wp:attachment":[{"href":"https:\/\/ckamath.org\/index.php\/wp-json\/wp\/v2\/media?parent=197"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/sampling_new.png\")
<\/figure>\n
\n\n\n\n\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/process_map_new.png\")
<\/figure>\n
\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/som_inverse_new.png\")
<\/figure>\n
\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/BlockGP_a_new.png\")
<\/figure>\n
\n\n\n\n\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/Compression_new.jpg\")
<\/figure>\n
\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/IntExpl_new.png\")
<\/figure>\n
\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/WindSOMS.png\")
<\/figure>\n
\n\n\n\n\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/featsel_image_new.jpg\")
<\/figure>\n
\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/aspen_new.jpg\")
<\/figure>\n
\n\n\n\n\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/Tracking_a_new.png\"){kind=tracking}
<\/figure>\n
\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/pde_new.jpg\")
<\/figure>\n
\n\n\n\n![\"\"](\"http:\/\/ckamath.org\/wp-content\/uploads\/2022\/03\/Sapphire_new.png\")
<\/figure>\n