The Landweber iteration or Landweber algorithm is an algorithm to solve ill-posed linear inverse problems, and it has been extended to solve non-linear problems that involve constraints. The method was first proposed in the 1950s by Louis Landweber, and it can be now viewed as a special case of many other more general methods. == Basic algorithm == The original Landweber algorithm attempts to recover a signal x from (noisy) measurements y. The linear version assumes that y = A x {\displaystyle y=Ax} for a linear operator A. When the problem is in finite dimensions, A is just a matrix. When A is nonsingular, then an explicit solution is x = A − 1 y {\displaystyle x=A^{-1}y} . However, if A is ill-conditioned, the explicit solution is a poor choice since it is sensitive to any noise in the data y. If A is singular, this explicit solution doesn't even exist. The Landweber algorithm is an attempt to regularize the problem, and is one of the alternatives to Tikhonov regularization. We may view the Landweber algorithm as solving: min x ‖ A x − y ‖ 2 2 / 2 {\displaystyle \min _{x}\|Ax-y\|_{2}^{2}/2} using an iterative method. The algorithm is given by the update x k + 1 = x k − ω A ∗ ( A x k − y ) . {\displaystyle x_{k+1}=x_{k}-\omega A^{}(Ax_{k}-y).} where the relaxation factor ω {\displaystyle \omega } satisfies 0 < ω < 2 / σ 1 2 {\displaystyle 0<\omega <2/\sigma _{1}^{2}} . Here σ 1 {\displaystyle \sigma _{1}} is the largest singular value of A {\displaystyle A} . If we write f ( x ) = ‖ A x − y ‖ 2 2 / 2 {\displaystyle f(x)=\|Ax-y\|_{2}^{2}/2} , then the update can be written in terms of the gradient x k + 1 = x k − ω ∇ f ( x k ) {\displaystyle x_{k+1}=x_{k}-\omega \nabla f(x_{k})} and hence the algorithm is a special case of gradient descent. For ill-posed problems, the iterative method needs to be stopped at a suitable iteration index, because it semi-converges. This means that the iterates approach a regularized solution during the first iterations, but become unstable in further iterations. The reciprocal of the iteration index 1 / k {\displaystyle 1/k} acts as a regularization parameter. A suitable parameter is found, when the mismatch ‖ A x k − y ‖ 2 2 {\displaystyle \|Ax_{k}-y\|_{2}^{2}} approaches the noise level. Using the Landweber iteration as a regularization algorithm has been discussed in the literature. == Nonlinear extension == In general, the updates generated by x k + 1 = x k − τ ∇ f ( x k ) {\displaystyle x_{k+1}=x_{k}-\tau \nabla f(x_{k})} will generate a sequence f ( x k ) {\displaystyle f(x_{k})} that converges to a minimizer of f whenever f is convex and the stepsize τ {\displaystyle \tau } is chosen such that 0 < τ < 2 / ( ‖ ∇ f ‖ 2 ) {\displaystyle 0<\tau <2/(\|\nabla f\|^{2})} where ‖ ⋅ ‖ {\displaystyle \|\cdot \|} is the spectral norm. Since this is special type of gradient descent, there currently is not much benefit to analyzing it on its own as the nonlinear Landweber, but such analysis was performed historically by many communities not aware of unifying frameworks. The nonlinear Landweber problem has been studied in many papers in many communities; see, for example. == Extension to constrained problems == If f is a convex function and C is a convex set, then the problem min x ∈ C f ( x ) {\displaystyle \min _{x\in C}f(x)} can be solved by the constrained, nonlinear Landweber iteration, given by: x k + 1 = P C ( x k − τ ∇ f ( x k ) ) {\displaystyle x_{k+1}={\mathcal {P}}_{C}(x_{k}-\tau \nabla f(x_{k}))} where P {\displaystyle {\mathcal {P}}} is the projection onto the set C. Convergence is guaranteed when 0 < τ < 2 / ( ‖ A ‖ 2 ) {\displaystyle 0<\tau <2/(\|A\|^{2})} . This is again a special case of projected gradient descent (which is a special case of the forward–backward algorithm) as discussed in. == Applications == Since the method has been around since the 1950s, it has been adopted and rediscovered by many scientific communities, especially those studying ill-posed problems. In X-ray computed tomography it is called simultaneous iterative reconstruction technique (SIRT). It has also been used in the computer vision community and the signal restoration community. It is also used in image processing, since many image problems, such as deconvolution, are ill-posed. Variants of this method have been used also in sparse approximation problems and compressed sensing settings.
Steerable filter
In image processing, a steerable filter is an orientation-selective filter that can be computationally rotated to any direction. Rather than designing a new filter for each orientation, a steerable filter is synthesized from a linear combination of a small, fixed set of "basis filters". This approach is efficient and is widely used for tasks that involve directionality, such as edge detection, texture analysis, and shape-from-shading. The principle of steerability has been generalized in deep learning to create equivariant neural networks, which can recognize features in data regardless of their orientation or position. == Example == A common example of a steerable filter is the first derivative of a two-dimensional Gaussian function. This filter responds strongly to oriented image features like edges. It is constructed from two basis filters: the partial derivative of the Gaussian with respect to the horizontal direction ( x {\displaystyle x} ) and the vertical direction ( y {\displaystyle y} ). If G ( x , y ) {\displaystyle G(x,y)} is the Gaussian function, and G x {\displaystyle G_{x}} and G y {\displaystyle G_{y}} are its partial derivatives (which measure the rate of change in the x {\displaystyle x} and y {\displaystyle y} directions, respectively), a new filter G θ {\displaystyle G_{\theta }} oriented at an angle θ {\displaystyle \theta } can be synthesized with the formula: G θ = cos ( θ ) G x + sin ( θ ) G y {\displaystyle G_{\theta }=\cos(\theta )G_{x}+\sin(\theta )G_{y}} Here, the basis filters G x {\displaystyle G_{x}} and G y {\displaystyle G_{y}} are weighted by cos ( θ ) {\displaystyle \cos(\theta )} and sin ( θ ) {\displaystyle \sin(\theta )} to "steer" the filter's sensitivity to the desired orientation. This is equivalent to taking the dot product of the direction vector ( cos θ , sin θ ) {\displaystyle (\cos \theta ,\sin \theta )} with the filter's gradient, ( G x , G y ) {\displaystyle (G_{x},G_{y})} . == Generalization in deep learning: Equivariant neural networks == The concept of steerability is foundational to equivariant neural networks, a class of models in deep learning designed to understand symmetries in data. A network is considered equivariant to a transformation (like a rotation) if transforming the input and then passing it through the network produces the same result as passing the input through the network first and then transforming the output. Formally, for a transformation T {\displaystyle T} and a network f {\displaystyle f} , this property is defined as f ( T ( input ) ) = T ( f ( input ) ) {\displaystyle f(T({\text{input}}))=T(f({\text{input}}))} . This built-in understanding of geometry makes models more data-efficient. For example, a network equivariant to rotation does not need to be shown an object in multiple orientations to learn to recognize it; it inherently understands that a rotated object is still the same object. This leads to better generalization and performance, particularly in scientific applications. === Mathematical foundation === Equivariant neural networks use principles from group theory to create operations that respect geometric symmetries, such as the SO(3) group for 3D rotations or the E(3) group for rotations and translations. Instead of learning standard filter kernels, these networks learn how to combine a fixed set of basis kernels. These basis functions are chosen so that they have well-defined behaviors under transformation groups. Spherical harmonics are frequently used as basis functions because they form a complete set of functions that behave predictably under rotation, making them ideal for creating steerable 3D kernels. Features within the network are treated as geometric tensors, which are mathematical objects (like scalars or vectors) that are "typed" by their behavior under transformations. These types correspond to the irreducible representations (irreps) of the group. The tensor product is the fundamental operation used to combine these typed features in a way that preserves equivariance, guaranteeing that the network as a whole respects the desired symmetry. Frameworks like e3nn simplify the construction of these networks by automating the complex mathematics of irreducible representations and tensor products. === Applications === Steerable and equivariant models are highly effective for problems with inherent geometric symmetries. Examples include: Protein structure analysis: SE(3)-equivariant networks can process 3D molecular structures while respecting their rotational and translational symmetries. 3D Point cloud processing: Rotation-equivariant filters built from steerable spherical functions can perform tasks like 3D shape classification. Computational chemistry: E(3)-equivariant graph neural networks are used to model interatomic potentials for molecular dynamics simulations, creating highly accurate and data-efficient models of physical systems.
Best AI Voice Assistants in 2026
Trying to pick the best AI voice assistant? An AI voice assistant is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI voice assistant slots into your workflow and pays for itself fast. This guide breaks down the top picks, their pros and cons, and who each one is best for.
Seppo Linnainmaa
Seppo Ilmari Linnainmaa (born 28 September 1945) is a Finnish mathematician and computer scientist known for creating the modern version of backpropagation. == Biography == He was born in Pori. He received his MSc in 1970 and introduced a reverse mode of automatic differentiation in his MSc thesis. In 1974 he obtained the first doctorate ever awarded in computer science at the University of Helsinki. In 1976, he became Assistant Professor. From 1984 to 1985 he was Visiting Professor at the University of Maryland, USA. From 1986 to 1989 he was Chairman of the Finnish Artificial Intelligence Society. From 1989 to 2007, he was Research Professor at the VTT Technical Research Centre of Finland. He retired in 2007. == Backpropagation == Explicit, efficient error backpropagation in arbitrary, discrete, possibly sparsely connected, neural networks-like networks was first described in Linnainmaa's 1970 master's thesis, albeit without reference to NNs, when he introduced the reverse mode of automatic differentiation (AD), in order to efficiently compute the derivative of a differentiable composite function that can be represented as a graph, by recursively applying the chain rule to the building blocks of the function. Linnainmaa published it first, following Gerardi Ostrowski who had used it in the context of certain process models in chemical engineering some five years earlier, but didn't publish.
Nicolò Cesa-Bianchi
Nicolò Cesa-Bianchi (Italian pronunciation: [nikoˈlɔ tˈtʃɛːza ˈbjaŋki]) is an Italian computer scientist and Professor of Computer Science at the Department of Computer Science of the University of Milan. He is a researcher in the field of machine learning, and co-author of the books "Prediction, Learning, and Games" with Gabor Lugosi and "Regret analysis of stochastic and nonstochastic multi-armed bandit problems" with Sébastien Bubeck == Education and career == Cesa-Bianchi graduated in Computer Science from the University of Milan in 1988 where he received a PhD in Computer Science in 1993 supervised by Alberto Bertoni. During his PhD, he visited UC Santa Cruz where he worked with Manfred Warmuth and David Haussler. He did his postdoctoral studies at Graz University of Technology under the supervision of Wolfgang Maass. == Research == His research contributions focus on the following areas: design and analysis of machine learning algorithms, especially in online machine learning algorithms for multi-armed bandit problems, with applications to recommender systems and online auctions graph analytics, with applications to social networks and bioinformatics == Awards and honors == Cesa-Bianchi received a Google Research Award in 2010, a Xerox University Affairs Committee Award in 2011, a Criteo Faculty Award in 2017, a Google Faculty Award in 2018, and a IBM Academic Award in 2021. Since 2023 he is corresponding member of the Accademia dei Lincei.
Grammar systems theory
Grammar systems theory is a field of theoretical computer science that studies systems of finite collections of formal grammars generating a formal language. Each grammar works on a string, a so-called sequential form that represents an environment. Grammar systems can thus be used as a formalization of decentralized or distributed systems of agents in artificial intelligence. Let A {\displaystyle \mathbb {A} } be a simple reactive agent moving on the table and trying not to fall down from the table with two reactions, t for turning and ƒ for moving forward. The set of possible behaviors of A {\displaystyle \mathbb {A} } can then be described as formal language L A = { ( f m t n f r ) + : 1 ≤ m ≤ k ; 1 ≤ n ≤ ℓ ; 1 ≤ r ≤ k } , {\displaystyle \mathbb {L_{A}} =\{(f^{m}t^{n}f^{r})^{+}:1\leq m\leq k;1\leq n\leq \ell ;1\leq r\leq k\},} where ƒ can be done maximally k times and t can be done maximally ℓ times considering the dimensions of the table. Let G A {\displaystyle \mathbb {G_{A}} } be a formal grammar which generates language L A {\displaystyle \mathbb {L_{A}} } . The behavior of A {\displaystyle \mathbb {A} } is then described by this grammar. Suppose the A {\displaystyle \mathbb {A} } has a subsumption architecture; each component of this architecture can be then represented as a formal grammar, too, and the final behavior of the agent is then described by this system of grammars. The schema on the right describes such a system of grammars which shares a common string representing an environment. The shared sequential form is sequentially rewritten by each grammar, which can represent either a component or generally an agent. If grammars communicate together and work on a shared sequential form, it is called a Cooperating Distributed (DC) grammar system. Shared sequential form is a similar concept to the blackboard approach in AI, which is inspired by an idea of experts solving some problem together while they share their proposals and ideas on a shared blackboard. Each grammar in a grammar system can also work on its own string and communicate with other grammars in a system by sending their sequential forms on request. Such a grammar system is then called a Parallel Communicating (PC) grammar system. PC and DC are inspired by distributed AI. If there is no communication between grammars, the system is close to the decentralized approaches in AI. These kinds of grammar systems are sometimes called colonies or Eco-Grammar systems, depending (besides others) on whether the environment is changing on its own (Eco-Grammar system) or not (colonies).
The Best Free AI Website Builder for Beginners
In search of the best AI website builder? An AI website builder is software that uses machine learning to help you get more done — it turns a rough idea into a polished result in seconds. When choosing one, weigh output quality, pricing, export formats, and how well it fits the tools you already use. Whether you are a beginner or a pro, the right AI website builder slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.