Convex Clustering and Lyapunov Functional Minimization in High Dimensional Optimization

282 https://researchtrendsjournal.com Online at: https://researchtrendsjournal.com ISSN No: 2584-282X Indexed Journal Peer Reviewed Journal INTERNATIONAL JOURNAL OF TRENDS IN EMERGING RESEARCH AND DEVELOPMENT Volume 2; Issue 6; 2024; Page No. 282-287 Received: 02-09-2024 Accepted: 09-10-2024 Convex Clustering and Lyapunov Functional Minimization in High Dimensional Optimization 1Rupesh Kumar and 2Dr. Brij Pal Singh 1Research Scholar, Department of Mathematics, Sunrise University, Alwar, Rajasthan, India 2Professor, Department of Mathematics, Sunrise University, Alwar, Rajasthan, India DOI: https://doi.org/10.5281/zenodo.17207023 Corresponding Author: Rupesh Kumar Abstract The piecewise constant Mumford-Shah model is used to investigate a multi-class segmentation issue inside a graph framework; this topic is pertinent to an adjacent area of study. We provide an effective strategy based on the MBO approach for the graph form of the MumfordShah model. In theoretical study, it is shown that when algorithm complexity increases, a Lyapunov functional decreases. Also, for big datasets, we estimate the eigenvectors of the graph Laplacian efficiently using a limited subset of the weight matrix using the Nyström extension technique, which helps to lower the computational cost. Finally, we apply the suggested method to the issue of chemical plume identification in hyper spectral video data. We presented graph-based clustering methods that drastically cut down on processing time for massive datasets. A straightforward and very parallelizable approach to multiway graph partitioning, our incremental reseeding clustering technique, is presented in the final chapter. We demonstrate via experiments that our method achieves top-notch results for cluster purity on common benchmark datasets. The method is also orders of magnitude faster than competing approaches. Keywords: Lyapunov, Functional, Laplacian, Eigen, Vectors, Magnitude Introduction Revolutionary harmonic optimizations and the discovery of simulated annealing both of which were influenced by the actual process of annealing are also part of our path. In order to tackle control engineering and parameter optimization issues, Particle Swarm Optimization (PSO) has hopped on the bandwagon. The Ant Colony Algorithm, which takes its cues from ant behaviors, is designed to fix scheduling and routing problems. Reason enough to advance in a number of domains More than that, scheduling and resource allocation become much easier when one delves into the realm of restricted programming, which deals with solving combinatorial problems with complicated restrictions. Optimization techniques have come a long way since the advent of inner point methods, which efficiently and accurately solve linear and nonlinear programming problems. Tobu searches with non-convex goals, a novel method that employs memory-based techniques to efficiently traverse complicated search spaces and achieve outstanding results in combinatorial optimizations issues, were also covered. We also see the truth of convex optimization, whose strength reveals its uses in signal processing, portfolio optimizations, and much besides. Tracker can assess the evolution and change of optimizations methods throughout time by doing this research and diving into the historical history of optimizations algorithms. The receiver may see the creation and evolution of optimizations approaches across time, as well as the impact of each algorithm in solving real-world issues and shaping advancements in other domains, by drawing a map of their contributions. Additionally, we want to provide a holistic view of the most significant optimizations algorithms, including their underlying concepts and the numerous domains in which they have had an influence. You can better evaluate the present level of optimizations algorithms and forecast the forthcoming developments in this dynamic field of research by studying their strengths, limits, and historical International Journal of Trends in Emerging Research and Development https://researchtrendsjournal.com 283 https://researchtrendsjournal.com background. An art gallery greets us at the entrance as we venture into the realm of optimizations algorithms, where we will uncover their influence on problem-solving. Convex Clustering Problem Formulation As a more efficient substitute for traditional clustering methods like k-means, convex clustering has grown in popularity. Nevertheless, convex clustering provides a more feasible alternative to the NP-hard K-Means technique, which requires the input of k before it can be used. To accomplish agglomerative clustering, Convex relaxed kmeans clustering, introduced by Lindsten et al., employs a fusion penalty. In addition, it provides a thorough examination of convex clustering, including several model formulations and optimizations techniques. By showcasing its statistical features and its many applications across several domains, this extensive overview deepens our theoretical and practical knowledge of convex clustering. Data and the shapes of the clusters are used to regulate convex clustering. You may do this by changing the value of the hyperparameter λ. The placements of the cluster’s centroids are controlled by this hyperparameter, which may have an effect on the cluster’s performance efficiency. To prevent solutions with insignificant outcomes, use the correct λ. Chi et al., Wang et al., Tan and Witten. are among the researchers who have investigated the details of choosing λ. By carefully assigning weights to the regularization term, weighted convex clustering takes the idea a step further. Stability against outliers and noise is improved by this change. Convex clustering is mathematically appealing, but it has problems in noisy situations since it needs perfect data features. It is worth mentioning that in this context, Prony; s technique has been investigated for possible enhancements by enhancing its robustness against noise perturbations using nuclear-norm penalized regularization. This method is frequently used for signal modelling and uses a finite sum of exponential components. To overcome convex clustering’s susceptibility to noise in high-dimensional datasets, several supplementary methods may provide useful insights. Literature Review Jaesoo et al. (2023) [1] In order This article gives three knearest neighbor (k-NN) optimization strategies for a distributed, in-memory, high-dimensional vector network to speed up content-based picture retrieval. index search. Three optimization methods have been proposed: one that executes k-NN optimization using data distribution, another that optimizes learning using cost statistics derived from query processing, and a third that utilizes a model trained using query logs for deep learning. Using in-memory highdimensional indexing, all three methods accomplish distributed k-NN queries. The spark was used to execute the suggested methods since it allows for distributed processing on a large scale using a master/slave approach. Using a battery of performance assessments grounded on highdimensional data, we proved the validity and superiority of the suggested methods. Lokesh et al. (2023) [2] The digitization of many sectors is generating enormous amounts of data in areas such as healthcare, retail, the web, and the web of things (IoT). We find data patterns for data attributes. using machine learning (ML) methods. Data is created at an exponential pace in today rapidly evolving world by individuals, robots, and businesses alike. As the need for computer scientists grows, more and more companies are investing in research that uses machine learning algorithms and sophisticated methodologies to transform large datasets containing disparate data types into meaningful patterns. Scientists are facing increasing challenges in effectively extracting valuable insights from the mountain of high-dimensional big data. When faced with massive data sets, traditional data mining techniques fail miserably. Predictive analytics is getting a lot of attention since big data is growing at an exponential rate. Integrating technologies that are datadriven with algorithms that use machine learning and predictive big data analytics allows for the evaluation of many data patterns and the investigation of both historical and future data based on these patterns. With the use of hyperparameter optimization and the dimension reduction approach, this research study proposes a methodology called splitting random forest (SRF) for predictive analysis on huge data. Xin et al. (2017) [3] Divide-and-Conquer (DC) is a good fit for high-dimensional optimization issues because it breaks the problem down into many smaller, more manageable problems and solves each one independently. However, considering the original issue and its subproblems have different dimensionalities makes it difficult to accurately evaluate possible answers to a set of smaller problems. Applying DC to non-separable high-dimensional optimization issues has shown to be somewhat challenging due to this. Finding a good solution to a sub-issue is a computationally expensive challenge that this paper suggests solving using meta-models. This led to the release of a new approach known as A process known as SelfEvaluation Evolution (SEE). As the size of the problem grows, empirical research conducted on the CEC2010 largescale global optimization benchmark reveals that SEE outperforms the four sample approaches. The empirical investigations also examine SEE limitations. Issa M.S.ALI Dr. B. Mukunthan et al. (2020) [5] Since the world is rapidly digitizing and there is an excess of data coming from all kinds of different places and formats, traditional systems just can’t handle the computational and analytical demands of big data. This is where open-source software like Hadoop comes in. In a partitioned setting, it keeps and processes data. Building Big Data Applications has grown in importance over the last decade. Getting to the knowledge essence in massive amounts of data is crucial for many organizations. Nevertheless, the performance, accuracy, responsiveness, and scalability of standard data techniques are diminished in their display. Much effort has gone into finding a solution to the complex Big Data challenge. Many different kinds of technology have been created for that purpose. A review of current optimization methods and their Big Data applications is the subject of this study. Its goal is to facilitate the selection of an appropriate Big Data technology partnership in order to meet the needs. Hemant et al. (2019) [6] Optimisation approaches in Operations Research (OR) are thoroughly examined in this review article, which aims to shed light on their relevance, current state of the art, obstacles, and potential future International Journal of Trends in Emerging Research and Development https://researchtrendsjournal.com 284 https://researchtrendsjournal.com developments. Optimisation of decision-making processes across diverse businesses is the goal of Operations Research, which employs mathematical and analytical methodologies. The first part of this page serves as an overview of OR, touching on its background, scope, and definition. After that, it gets down to brass tacks, discussing various optimisation challenges and strategies. The article begins with a discussion of classical optimisation methods like linear and integer programming as well as nonlinear programming. It then moves on to heuristic and metaheuristic methods such as optimization for ant colonies, optimization for particle swarms, and tabu search, simulated annealing, and genetic algorithms. A number of new optimisation techniques, such as hybrid approaches, optimisation for large data, optimisation with multiple objectives, and integration with AI and machine learning. In addition, new developments and potential future directions are discussed in the article, along with the difficulties encountered by optimisation research. This work seeks to make a contribution to the continuous improvement and implementation of optimisation methods in OR and other domains by consolidating existing information and highlighting potential avenues for further study. Research Methodology Modularity with Multiple Slices Equation which evaluates the modularity Q, is a quality function that assesses the & quot; goodness & quot; of a clustering, as mentioned in the previous chapter. One way to investigate the community structure of a network is to find a division that aims to maximize Q. Segmenting a network into communities of varying sizes is made possible by modularity optimisation, which differs from conventional spectral clustering methods in that it doesnt depend on knowing the number or size of communities. Fig 1: Schematic of a multislice network. Community structure in countless networks and hyperspectral image analysis have both made use of optimisation of the usual modularity function (2.12). Here, we optimize multislice modularity (3.1) for the purpose of studying image segmentation and social network community structure. We begin with a static graph in every instance, and the multi-slice network employs the same adjacency matrix with varying resolutionparameter values for each layer γ s. We just consider the edges between neighboring slices while considering each node j, therefore C jsr = 0 unless |r − s| = 1. Every inter-slice edge that is not zero is fixed to a constant value ω. Social and geographic matrix Establish a similarity (weight) matrix W that combines the geographic data G and the social relationship S in a linear fashion: Results Mumford-Shah MBO and Lyapunov functional: To estimate motion in Euclidean space by the mean curvature of an interface, the authors of provide an efficient technique called the MBO scheme. The MBO scheme overall process iteratively switches between thresholding and solving a linear heat equation. According to one theory, the technique uses thresholding in lieu of the nonlinear component of the Allen-Cahn equation. To find the energy minimizer as close as possible, we provide here an alternative to the original MBO plan. We derive a Lyapunov functional Y (f) for our algorithm from the research of and demonstrate that it lowers with each iteration of the MBO scheme. Numerical Results Graph Laplacian eigenvectors for normalized cut energy optimization. The initial input similarity matrix is factorized and regularized using the NMFR method using graph-based random walk principles and non-negative matrix factorization. One such method for factoring matrices that do not include negative entries is the LSD algorithm. The goal is to attend to a similarity matrix decomposition that is stochastic on the left. Using L1 optimization approaches, the MTV algorithm from aims to findan optimum multiway Cheeger cut of the network. It is a total-variation based algorithm. In order to get an initial partition, the final three algorithms-NMFR, LSD, and MTV-all employ NCut. We International Journal of Trends in Emerging Research and Development https://researchtrendsjournal.com 285 https://researchtrendsjournal.com use a random partition to start our approach, in contrast. We use the NCut code from, the NMFR and LSD test codes from, and the MTV algorithm code from for our nonnegative matrix factorization techniques. Table 1: Algorithmic Comparison via Cluster Purity. Data size R RND NCut LSD NMFR MTV INCRES (speed 1) INCRES (speed 5) 20 News 20K 20 6.3% 26.6% 34.3% 60.7% 35.8% 61.0% 60.7% Cade 2IK 3 15.5% 41.0% 41.3% 52.0% 44.2% 52.8% 52.1% Rcv1 9.6K 4 30.3% 38.2% 38.1% 42.7% 42.8% 54.5% 51.2% Webkb4 4.2K 4 39.1% 39.8% 45.8% 58.1% 45.2% 57.1% 56.8% Citeseer 3.3K 6 21.8% 23.4% 53.4% 62.6% 42.6% 62.0% 62.2% Mnist 70K 10 11.3% 76.9% 75.5% 97.1% 95.5% 96.0% 94.0% Pendigit 11K 10 11.6% 80.2% 86.1% 86.8% 86.5% 88.8% 85.9% Usps 9.3K 10 16.7% 71.5% 70.4% 86.4% 85.3% 87.8% 87.4% Optdigit 5.6K 10 12.0% 90.8% 91.0% 98.0% 95.2% 97.4% 94.8% Speed comparisons The rate of convergence towards solutions is shown in Figure 2. for the iterative algorithms LSD, MTV, NMFR, and INCRES. We gave 20NEWS and MNIST each method a total of seven and fifteen minutes, respectively. We document the algorithm purity output at each iteration. By averaging the results across 240 runs, the purity curves for the randomized algorithms (INCRES and MTV) were produced. All of these techniques are very computationally intensive since at each step you have to multiply a sparse matrix by a complete matrix. All tests were conducted on the same architecture, and each method is implemented fairly and consistently. Fig 2: Purity curves for the four algorithms considered on two benchmark data sets (20NEWS and MNIST). International Journal of Trends in Emerging Research and Development https://researchtrendsjournal.com 286 https://researchtrendsjournal.com Table 2: Robustness Comparisons Noise N Cut LSD MTV INCRES (speed1) 20 NEWS +0% NEWS 27% 34% 36% 61% +50% edges 21% 27% 20% 52% +100% edges 18% 22% 11% 44% +150% edges 15% 20% 10% 34% +200% edges 14% 18% 9% 27% MNIST +0% edges 77% 76% 96% 96% +50% edges 87% 94% 55% 97% +100% edges 84% 93% 25% 97% +150% edges 74% 87% 18% 97% +200% edges 67% 82% 16% 96% Privacy Preservation The primary focus of our algorithms for the study agents is safeguarding summary statistics and personally identifiable information; for example, agent j objective is to ensure the security of each record of which will from now on be called individual subject data, and any statistic derived from. Our schemes are based on the premise that agents safeguard personal data by keeping it in a secure area that is accessible only to that agent. Put another way, no other agent in the scheme can see any other agent personal information. All agents, both within and outside the system, have access to any data transferred between them under the & quot; honest but curious & quot; premise. The term & quot; privacy" is defined in this context as the absence of disclosure of personally identifiable information or aggregated statistical information about an individual in the event that agent A communication is overheard. Fig 3: Illustration of Centralized Scheme. Each arrow denotes the transmission of encoded data that can only be decoded by the global center after receiving J encoded aggregates from the J computation centers. Conclusion This chapter introduces a graphical framework for the multiclass piecewise constant Mumford-Shah model using a simplex-constrained representation. We present an effective threshold dynamics approach, the Mumford-Shah MBO scheme, for addressing the minimization issue based on the graph model. Theoretical research demonstrates that the MBO iteration reduces a Lyapunov energy that approximates the MS functional. Additionally, to diminish the computational expense associated with huge datasets, we use the Nyström extension technique to roughly calculate a subset of the eigenvectors of the normalized graph Laplacian, therefore obviating the need to compute the whole weight matrix of the graph. Upon acquiring the eigenvectors, each iteration of the Mumford-Shah MBO method has a temporal complexity of O(n). Empirically, the number of iterations required for convergence is minimal. The suggested approach is applicable to generic highdimensional data segmentation issues. This study focusses on the segmentation of hyperspectral video data. Numerical tests are conducted on a set of hyper-spectral pictures obtained from a movie for plume identification; our suggested technique yields competitive results. Nonetheless, there were unresolved enquiries to be addressed. The Nyström technique only computes eigenvectors for the normalized Laplacian, while the theoretical study of the Lyapunov functional is applicable just to the unnormalized graph Laplacian. This matter requires more examination. It is important to note that the graph created in this study incorporates just the spectral information of the pixels, excluding their spatial information. It is indeed possible to construct a graph that includes the position of each pixel, hence creating a non-local means graph as previously mentioned. International Journal of Trends in Emerging Research and Development https://researchtrendsjournal.com 287 https://researchtrendsjournal.com References 1. Park JC. The Historical Development and Characteristics of Courses at Korean Universities. Chongkyo yŏnku. 2023;83(1):65-100. 2. Manjunatha Swamy C, Sundaram SM, Lokesh MR. Performance analysis of feature selection and classification in Big Data Information extraction. Saudi J Eng Technol. 2023;8(3):62-70. 3. Krishnadoss N, Ramasamy L. A study on high dimensional big data using predictive data analytics model. Indonesian Journal of Electrical Engineering and Computer Science. 2023;30:174–182. doi:10.11591/ijeecs.v30.i1.pp174-182. 4. Xin Y, Yu YX. Quantum capacitance, electrostatic potential, electronic and structural data for bare and functionalized niobium carbide MXenes. Data in brief. 2017;15:623. 5. Ali IM, Mukunthan B. Big data optimization techniques: an empirical study. International Journal of Scientific and Technology Research. 2020;9(3):1-6. ISSN: 2277-8616. 6. Sonawane T, Azaz S, Hemant K, Liji T. Epididymal toxicity associated with vincristine treatment. Indian Journal of Pharmaceutical Sciences. 2019;81(3):514520. 7. Kalina J. High-dimensional data in economics and their (robust) analysis. Serbian Journal of Management. 2016;12. doi:10.5937/sjm12-10778. 8. Emrouznejad A. Big data optimization: recent developments and challenges. In: Big Data Optimization. Cham: Springer; c2016. doi:10.1007/978-3-319-30265-2. 9. Roy C, Rautaray S, Pandey M. Big data optimization techniques: a survey. International Journal of Information Engineering and Electronic Business. 2018;10(4):41–48. doi:10.5815/ijieeb.2018.04.06. 10. Ju Y, Qin R, Kipouros T, Parks G, Zhang C. A highdimensional design optimisation method for centrifugal impellers. Proceedings of the Institution of Mechanical Engineers, Part A: Journal of Power and Energy. 2016. p. 230. doi:10.1177/0957650915626274. Creative Commons (CC) License This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY 4.0) license. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.