Control de sistemas usando aprendizaje de máquina

Systems control using machine learning

Jesús Martín Miguel Martínez (2023)

El aprendizaje por refuerzo es un paradigma del aprendizaje de máquina con un amplio desarrollo y una creciente demanda en aplicaciones que involucran toma de decisiones y control. Es un paradigma que permite el diseño de controladores que no dependen directamente del modelo que describe la dinámica del sistema. Esto es importante ya que en aplicaciones reales es frecuente que no se disponga de dichos modelos de manera precisa. Esta tesis tiene como objetivo implementar un controlador óptimo en tiempo discreto libre de modelo. La metodología elegida se basa en algoritmos de aprendizaje por refuerzo, enfocados en sistemas con espacios de estado y acción continuos a través de modelos discretos. Se utiliza el concepto de función de valor (Q-función y función V ) y la ecuación de Bellman para resolver el problema del regulador cuadrático lineal para un sistema mecánico masa-resorte-amortiguador, en casos donde se tiene conocimiento parcial y desconocimiento total del modelo. Para ambos casos las funciones de valor son definidas explícitamente por la estructura de un aproximador paramétrico, donde el vector de pesos del aproximador es sintonizado a través de un proceso iterativo de estimación de parámetros. Cuando se tiene conocimiento parcial de la dinámica se usa el método de aprendizaje por diferencias temporales en un entrenamiento episódico, que utiliza el esquema de mínimos cuadrados con mínimos cuadrados recursivos en la sintonización del crítico y descenso del gradiente en la sintonización del actor, el mejor resultado para este esquema es usando el algoritmo de iteración de valor para la solución de la ecuación de Bellman, con un resultado significativo en términos de precisión en comparación a los valores óptimos (función DLQR). Cuando se tiene desconocimiento de la dinámica se usa el algoritmo Q-learning en entrenamiento continuo, con el esquema de mínimos cuadrados con mínimos cuadrados recursivos y el esquema de mínimos cuadrados con descenso del gradiente. Ambos esquemas usan el algoritmo de iteración de política para la solución de la ecuación de Bellman, y se obtienen resultados de aproximadamente 0.001 en la medición del error cuadrático medio. Se realiza una prueba de adaptabilidad considerando variaciones que puedan suceder en los parámetros de la planta, siendo el esquema de mínimos cuadrados con mínimos cuadrados recursivos el que tiene los mejores resultados, reduciendo significativamente ...

Reinforcement learning is a machine learning paradigm with extensive development and growing demand in decision-making and control applications. This technique allows the design of controllers that do not directly depend on the model describing the system dynamics. It is useful in real-world applications, where accurate models are often unavailable. The objective of this work is to implement a modelfree discrete-time optimal controller. Through discrete models, we implemented reinforcement learning algorithms focused on systems with continuous state and action spaces. The concepts of value-function, Q-function, V -function, and the Bellman equation are employed to solve the linear quadratic regulator problem for a mass-spring-damper system in a partially known and utterly unknown model. For both cases, the value functions are explicitly defined by a parametric approximator’s structure, where the weight vector is tuned through an iterative parameter estimation process. When partial knowledge of the dynamics is available, the temporal difference learning method is used under episodic training, utilizing the least squares with a recursive least squares scheme for tuning the critic and gradient descent for the actor´s tuning. The best result for this scheme is achieved using the value iteration algorithm for solving the Bellman equation, yielding significant improvements in approximating the optimal values (DLQR function). When the dynamics are entirely unknown, the Q-learning algorithm is employed in continuous training, employing the least squares with recursive least squares and the gradient descent schemes. Both schemes use the policy iteration algorithm to solve the Bellman equation, and the system’s response using the obtained values was compared to the one using the theoretical optimal values, yielding approximately zero mean squared error between them. An adaptability test is conducted considering variations that may occur in plant parameters, with the least squares with recursive least squares scheme yielding the best results, significantly reducing the number of iterations required for convergence to optimal values.

Master thesis

aprendizaje por refuerzo, control óptimo, control adaptativo, sistemas mecánicos, libre de modelo, dinámica totalmente desconocida, aproximación paramétrica, Q-learning, iteración de política reinforcement learning, optimal control, adaptive control, mechanical systems, modelfree, utterly unknown dynamics, parametric approximation, Q-learning, policy iteration INGENIERÍA Y TECNOLOGÍA CIENCIAS TECNOLÓGICAS TECNOLOGÍA DE LOS ORDENADORES INTELIGENCIA ARTIFICIAL INTELIGENCIA ARTIFICIAL

Stability of FeVO4-II under Pressure: A First-Principles Study

PRICILA BETBIRAI ROMERO VAZQUEZ SINHUE LOPEZ MORENO Daniel Errandonea (2022)

"In this work, we report first-principles calculations to study FeVO4 in the CrVO4-type (phase II) structure under pressure. Total-energy calculations were performed in order to analyze the structural parameters, the electronic, elastic, mechanical, and vibrational properties of FeVO4-II up to 9.6 GPa for the first time. We found a good agreement in the structural parameters with the experimental results available in the literature. The electronic structure analysis was complemented with results obtained from the Laplacian of the charge density at the bond critical points within the Quantum Theory of Atoms in Molecules methodology. Our findings from the elastic, mechanic, and vibrational properties were correlated to determine the elastic and dynamic stability of FeVO4-II under pressure. Calculations suggest that beyond the maximum pressure covered by our study, this phase could undergo a phase transition to a wolframite-type structure, such as in CrVO4 and InVO4."

Article

FeVO4 under pressure CrVO4-type structure First-principles Mechanical properties Vibrational properties Electronic properties CIENCIAS FÍSICO MATEMÁTICAS Y CIENCIAS DE LA TIERRA FÍSICA FÍSICA DEL ESTADO SÓLIDO CRISTALOGRAFÍA CRISTALOGRAFÍA

Tensile behavior of 3D printed polylactic acid (PLA) based composites reinforced with natural fiber

Eliana M Agaliotis BALTAZAR DAVID AKE CONCHA ALEJANDRO MAY PAT Juan Pablo Morales Arias Celina Bernal Alex Valadez González Pedro Jesús Herrera Franco Gwenaelle Proust JUAN FRANCISCO KOH DZUL José Gonzalo Carrillo Baeza Emmanuel Alejandro Flores Johnson (2022)

Natural fiber-reinforced composite (NFRC) filaments for 3D printing were fabricated using polylactic acid (PLA) reinforced with 1–5 wt% henequen flour comprising particles with sizes between 90–250 μm. The flour was obtained from natural henequen fibers. NFRCs and pristine PLA specimens were printed with a 0° raster angle for tension tests. The results showed that the NFRCs’ measured density, porosity, and degree of crystallinity increased with flour content. The tensile tests showed that the NFRC Young’s modulus was lower than that of the printed pristine PLA. For 1 wt% flour content, the NFRCs’ maximum stress and strain to failure were higher than those of the printed PLA, which was attributed to the henequen fibers acting as reinforcement and delaying crack growth. However, for 2 wt% and higher flour contents, the NFRCs’ maximum stress was lower than that of the printed PLA. Microscopic characterization after testing showed an increase in voids and defects, with the increase in flour content attributed to particle agglomeration. For 1 wt% flour content, the NFRCs were also printed with raster angles of ±45° and 90° for comparison; the highest tensile properties were obtained with a 0° raster angle. Finally, adding 3 wt% content of maleic anhydride to the NFRC with 1 wt% flour content slightly increased the maximum stress. The results presented herein warrant further research to fully understand the mechanical properties of printed NFRCs made of PLA reinforced with natural henequen fibers. © 2022 by the authors.

Article

POLYLACTIC ACID (PLA) NATURAL FIBER HENEQUEN FIBER NATURAL FIBER REINFORCED COMPOSITE (NFRC) ADDITIVE MANUFACTURING 3D PRINTING MECHANICAL PROPERTY INGENIERÍA Y TECNOLOGÍA CIENCIAS TECNOLÓGICAS TECNOLOGÍA DE MATERIALES PROPIEDADES DE LOS MATERIALES PROPIEDADES DE LOS MATERIALES

Sistemas de producción sostenibles y redes de innovación

Jelle Van Loon (2022)

Conference object

CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA PRODUCTION SYSTEMS INNOVATION PRODUCTION SYSTEMS AGRIFOOD SYSTEMS

Last mile seed delivery approaches in Sudan and beyond: an annotated bibliography

Hugo De Groote Paswel Marenya (2023)

Book

CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA SEED SYSTEMS SEED INDUSTRY BIBLIOGRAPHIES

Sistemas agroalimentarios sustentables: aumento de rentabilidad con uso eficiente de insumos y preservación del medio ambiente

Jelle Van Loon (2022)

Conference object

CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA AGRIFOOD SYSTEMS SUSTAINABLE INTENSIFICATION COVID-19 CONFLICTS CLIMATE CHANGE

Gender analysis of household seed security : A case of maize and wheat seed systems in Nepal

Hom Nath Gartaula (2022)

Book

Seed Security Mountains CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA SEED SYSTEMS MAIZE WHEAT ROLE OF WOMEN WOMEN'S PARTICIPATION

Women, economic resilience, gender norms in a time of climate change: what do we know?

Cathy Farnworth Anne Rietveld Rachel Voss Angela Meentzen (2023)

This literature delves into 82 research articles, published between 2016 and 2022, to develop a deep understanding of how women manage their lives and livelihoods within their agrifood systems when these systems are being affected, sometimes devastatingly, by climate change. The Findings show that four core gender norms affect the ability of women to achieve economic resilience in the face of climate change operate in agrifood production systems. Each of these gender norms speaks to male privilege: (i) Men are primary decision-makers, (ii) Men are breadwinners, (iii) Men control assets, and (iv) Men are food system actors. These gender norms are widely held and challenge women’s abilities to become economically resilient. These norms are made more powerful still because they fuse with each other and act on multiple levels, and they serve to support other norms which limit women’s scope to act. It is particularly noteworthy that many institutional actors, ranging from community decision-makers to development partners, tend to reinforce rather than challenge gender norms because they do not critically review their own assumptions.

However, the four gender norms cited are not hegemonic. First, there is limited and intriguing evidence that intersectional identities can influence women’s resilience in significant ways. Second, gender norms governing women’s roles and power in agrifood systems are changing in response to climate change and other forces, with implications for how women respond to future climate shocks. Third, paying attention to local realities is important – behaviours do not necessarily substantiate local norms. Fourth, women experience strong support from other women in savings groups, religious organisations, reciprocal labour, and others. Fifth, critical moments, such as climate disasters, offer potentially pivotal moments of change which could permit women unusually high levels of agency to overcome restrictive gender norms without being negatively sanctioned. The article concludes with recommendations for further research.

Article

Economic Resilience Intersectional Identities Women Groups Support CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA ECONOMICS RESILIENCE CLIMATE CHANGE GENDER NORMS AGRIFOOD SYSTEMS WOMEN

Bundling subsurface drip irrigation with no-till provides a window to integrate mung bean with intensive cereal systems for improving resource use efficiency

Manish Kakraliya madhu choudhary Mahesh Gathala Parbodh Chander Sharma ML JAT (2024)

The future of South Asia’s major production system (rice–wheat rotation) is at stake due to continuously aggravating pressure on groundwater aquifers and other natural resources which will further intensify with climate change. Traditional practices, conventional tillage (CT) residue burning, and indiscriminate use of groundwater with flood irrigation are the major drivers of the non-sustainability of rice–wheat (RW) system in northwest (NW) India. For designing sustainable practices in intensive cereal systems, we conducted a study on bundled practices (zero tillage, residue mulch, precise irrigation, and mung bean integration) based on multi-indicator (system productivity, profitability, and efficiency of water, nitrogen, and energy) analysis in RW system. The study showed that bundling conservation agriculture (CA) practices with subsurface drip irrigation (SDI) saved ~70 and 45% (3-year mean) of irrigation water in rice and wheat, respectively, compared to farmers’ practice/CT practice (pooled data of Sc1 and Sc2; 1,035 and 318 mm ha−1). On a 3-year system basis, CA with SDI scenarios (mean of Sc5–Sc8) saved 35.4% irrigation water under RW systems compared to their respective CA with flood irrigation (FI) scenarios (mean of Sc3 and Sc4) during the investigation irrespective of residue management. CA with FI system increased the water productivity (WPi) and its use efficiency (WUE) by ~52 and 12.3% (3-year mean), whereas SDI improved by 221.2 and 39.2% compared to farmers practice (Sc1; 0.69 kg grain m−3 and 21.39 kg grain ha−1 cm−1), respectively. Based on the 3-year mean, CA with SDI (mean of Sc5–Sc8) recorded −2.5% rice yield, whereas wheat yield was +25% compared to farmers practice (Sc1; 5.44 and 3.79 Mg ha−1) and rice and wheat yield under CA with flood irrigation were increased by +7 and + 11%, compared to their respective CT practices. Mung bean integration in Sc7 and Sc8 contributed to ~26% in crop productivity and profitability compared to farmers’ practice (Sc1) as SDI facilitated advancing the sowing time by 1 week. On a system basis, CA with SDI improved energy use efficiency (EUE) by ~70% and partial factor productivity of N by 18.4% compared to CT practices. In the RW system of NW India, CA with SDI for precise water and N management proved to be a profitable solution to address the problems of groundwater, residue burning, sustainable intensification, and input (water and energy) use with the potential for replication in large areas in NW India.

Article

Direct Seeded Rice Subsurface Drip Irrigation Economic Profitability Energy and Nitrogen Efficiency CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA CONSERVATION AGRICULTURE RICE SUBSURFACE IRRIGATION IRRIGATION SYSTEMS WATER PRODUCTIVITY ECONOMIC VIABILITY ENERGY EFFICIENCY NITROGEN-USE EFFICIENCY

Targeted nitrogen management to increase cereal production while reducing nitrogen consumption in India

Tek Sapkota Sieglinde Snapp (2022)

Conference object

CIENCIAS AGROPECUARIAS Y BIOTECNOLOGÍA CEREAL PRODUCTS PRODUCTION SYSTEMS CEREALS NITROGEN RICE WHEAT MAIZE