Extrapolation: initial commit

Research/Extrapolation methods/README.txt

+%--Introduction
+
+This research project is an extension of the ASPN investigations looking for cycling behaviour in Schwarz methods preconditioned by Newton-Raphson.
+Acceleration can be done by extrapolating from previous results, often improving speed up to quadratic convergence without significant increase in cost.
+Some of this extrapolation can be posed as CG or other Krylov subspace methods.
+
+Keywords:
+	--Method of Richardson
+	--Modified polynomial extrapolation (MPE)
+	--Conjugate gradient and Krylov subspace
+
+Goals:	--Arrive at a different method from Newton's (?)
+	--When is convergence quadratic?
+	--Apply such a method to the altS cyclic example
+	--Can all extrapolation methods be posed under a single unifying framework?
+
+%--Images
+
+Images are expected to be generated primarily through MATLAB experiments and TIKZ.
+They will follow the naming convention:
+	FORMAT_extrap_SpecificTopic_v1_YYYYMMDD.svg
+FORMAT: 	3 to 4 letter indicator of how the file was created, ie. MTLB (MATLAB) or TIKZ.
+extrap: 	indicates image is related to extrapolation research project.
+SpecificTopic: 	identifies topic within research project the image relates to.
+v1: 		version number.
+YYYYMMDD: 	date of initial creation.
+.svg:		file format; use .eps for vector graphics when possible.
+
+%--Code
+
+Code is expected to be almost entirely MATLAB.
+	TYPE_extrap_Function_v1_YYYYMMDD.m
+TYPE:		3 to 4 letters denoting intended result of the script:
+			PLOT: produces a plot, MTLB_extrap_Function_v1_YYYYMMDD.eps;
+			SUB: subfunction, used in another function;
+			EXP: computational experiment/test/example problem;
+			ALGO: implementation of algorithm.
+extrap:		indicates code relates to extrapolation research project.
+Function:	name to indicate related topic or result.
+v1:		version number.
+YYYYMMDD:	date of initial creation.
+.m:		file format; .m is for MATLAB files.
+
+%--LaTeX
+
+Notes and results will be typed in LaTeX.
+Both types of documents will use the custom MathArticle template.
+This provides a list of commands I seem to use often as well as the packages I tend to need.
+Since this does not come from a style file any changes can be made to suit the individual document's needs.
+
+A references.bib file will be created especially for this project, generated by Mendeley most likely.
+Mendeley uses as its preferred citation key AuthorYYYY.
+My preferred citation key is authorYYYYfirst, where first is the first word of the paper cited.
\ No newline at end of file

Research/Extrapolation methods/notes.tex

+\documentclass{article}
+
+\usepackage{amsmath}
+\usepackage{amssymb}
+\usepackage{amsthm}
+\usepackage{graphicx}
+\usepackage{hyperref}
+\usepackage{subcaption}
+
+\newcommand{\dxdy}[2]{\frac{d #1}{d #2}}
+\newcommand{\dxdyk}[3]{\frac{d^{#3} #1}{d {#2}^{#3}}}
+\newcommand{\pdxdy}[2]{\frac{\partial #1}{\partial #2}}
+\newcommand{\liminfty}[1]{\lim_{#1 \to \infty}}
+\newcommand{\limab}[2]{\lim_{#1 \to #2}}
+
+\newcommand{\abs}[1]{\left \vert #1 \right \vert}
+\newcommand{\norm}[1]{\left \Vert #1 \right \Vert}
+\newcommand{\order}[1]{\mathcal{O} \left ( #1 \right )}
+\newcommand{\set}[1]{\left \{ #1 \right \}}
+\newcommand{\Set}[2]{\left \{ #1 \ \middle \vert \ #2 \right \}}
+\newcommand{\vmat}[1]{\begin{vmatrix} #1 \end{vmatrix}}
+\DeclareMathOperator{\sign}{sign}
+
+\newcommand{\bbn}{\mathbb{N}}
+\newcommand{\bbz}{\mathbb{Z}}
+\newcommand{\bbq}{\mathbb{Q}}
+\newcommand{\bbr}{\mathbb{R}}
+\newcommand{\bbc}{\mathbb{C}}
+\newcommand{\bbf}{\mathbb{F}}
+
+\newtheorem{lemma}{Lemma}
+\newtheorem{thm}{Theorem}
+\newtheorem{cor}{Corollary}
+\newtheorem{prop}{Proposition}
+\newtheorem{conj}{Conjecture}
+\newtheorem{defn}{Definition}
+
+\begin{document}
+
+Nonlinear preconditioning: Krylov as extrapolation methods.
+There exists proof of quadratic convergence (where? copy here)
+
+Method of Richardson:
+\begin{equation*}
+	u^{k+1} = u^k + \frac{1}{\alpha_k} (f-Au^k)
+\end{equation*}
+where $\alpha_k$ is a parameter of the acceleration.
+This leads to a formula for the errors (?)
+\begin{equation*}
+	e_k^{n+1} = \dots = \Pi_{j=0}^n \left ( 1 - \frac{\lambda_k}{\alpha_j} \right ) e_k^0.
+\end{equation*}
+Choose $\alpha_j$ to minimize the polynomial.
+This leads to Chebyshev polynomials.
+
+Need to know spectrum.
+Alternative: take linear combinations of previous iterates.
+\begin{align*}
+e^n = & u - u^n \\
+\sum \gamma_j u^j = & u \sum \gamma_j + \sum \gamma_j e^j \\
+v^n = & u + \sum \gamma_j e^j \\
+u - v^n = \sum \gamma_j e^j
+\end{align*}
+and the goal is to minimize the last line.
+Set $d^n = u^{n+1} - u^n$ and
+\begin{align*}
+u = & M^{-1} N u + M^{-1} f \\
+u^{n+1} = & M^{-1} N u^{n-1} + M^{-1} f \\
+e^{n+1} = & u - u^{n+1} = M^{-1} N e^n \\
+\implies \sum \gamma_j e^j = & \sum \gamma_j G^j e^0 = p_n(G) e^0 \\
+d^n = & G d^{n-1} = G^n d^0 \\
+\sum \gamma_j d^j = & \sum \gamma_j G^j d^0 = p_n(G) d^0.
+\end{align*}
+Minimizing this last equation is called Modified Polynomial Extrapolation.
+
+Overdetermine the system:
+\begin{equation*}
+\sum \tilde{\gamma}_j d^j = -d^n
+\end{equation*}
+which leads to Krylov (thm, where? copy).
+
+Try to get to different method from Newton.
+Which methods have quadratic convergence?
+Apply methods to altS example.
+
+References to look into:
+\begin{itemize}
+\item book with Walter and Felix
+\item Brezinski (book)
+\item Sidi, sequence of papers
+\end{itemize}
+
+\end{document}
\ No newline at end of file