Integration: Kinematics + Digital Image Processing + 3D Graphics | ||
|
This article promotes integration idea as well as a lot of my other articles. This idea is illustrated by following problem. We would like visualize 3D object which has texture obtained by digital image processing.
All problems of article's title need data exchange. Data exchange architecture has four basic interfaces. First interface is elementary unit of data exchange.
/// <summary> /// Elementary unit of data exchange /// </summary> public interface IMeasure { /// <summary> /// Function which returns unit of data /// </summary> Func<object> Parameter { get; } /// <summary> /// The name of data unit /// </summary> string Name { get; } /// <summary> /// Type of parameter /// </summary> object Type { get; } }
Members of this interface are presented in following table.
N | Member name | Meaning |
1 | Parameter |
Function which returns unit of data |
2 | Name |
The name of data unit |
3 | Type |
Type of parameter |
The Parameter
function can return any object. The
Name
property enables us to link input parameters to
output ones. The Type
property contains Meta information about
unit. This property is not necessary variable of System.Type
type.
Following table is a mapping between variables and their "types".
N | Type of variable | Value of Type property |
1 | string |
string s = "" |
2 | bool |
bool b = false |
3 | double |
double d = 0 |
4 | float |
float f = 0 |
5 | byte |
byte b = 0x0 |
6 | sbyte |
sbyte s = 0x0 |
7 | short |
short s = 0 |
8 | short |
ushort u = 0 |
9 | int |
int i = 0 |
10 | uint |
uint u = 0 |
11 | long |
long l = 0 |
12 | ulong |
ulong u = 0 |
13 | Array |
ArrayReturnType a = (depends on type and length of
array) |
If elementary unit parameter is array then "type" of unit is
ArrayReturnType
variable. The
lArrayReturnType
code is presented below.
/// <summary> /// Return type of array /// </summary> public class ArrayReturnType { #region Fields object elementType; int[] dimension; bool isObjectType; #endregion /// <summary> /// Constructor /// </summary> /// <param name="elementType">Type of element</param> /// <param name="dimension">Dimension</param> /// <param name="objectType">The "is object" sign</param> public ArrayReturnType(object elementType, int[] dimension, bool objectType) { this.elementType = elementType; this.dimension = dimension; this.isObjectType = objectType; } /// <summary> /// The "is object" sign /// </summary> public bool IsObjectType { get { return isObjectType; } } /// <summary> /// Dimension /// </summary> public int[] Dimension { get { return dimension; } } /// <summary> /// Type of element /// </summary> public object ElementType { get { return elementType; } } /// <summary> /// Overriden Equals /// </summary> /// <param name="obj">Compared obje</param> /// <returns>True if equal</returns> public override bool Equals(object obj) { if (!(obj is ArrayReturnType)) { return false; } ArrayReturnType at = obj as ArrayReturnType; if (!at.elementType.Equals(elementType)) { return false; } if (at.dimension.Length != dimension.Length) { return false; } for (int i = 0; i < dimension.Length; i++) { if (dimension[i] != at.dimension[i]) { return false; } } return true; } /// <summary> /// Overriden /// </summary> /// <returns>Hash code</returns> public override int GetHashCode() { return dimension.Length * dimension[0]; } /// <summary> /// Checks equality of imension /// </summary> /// <param name="type">Type</param> /// <returns>True in case of equal dimesion and false otherwise</returns> public bool HasEqualDimension(ArrayReturnType type) { if (type.dimension.Length != dimension.Length) { return false; } for (int i = 0; i < dimension.Length; i++) { if (dimension[i] != type.dimension[i]) { return false; } } return true; } /// <summary> /// Gets base type /// </summary> /// <param name="o">The object</param> /// <returns>Object's base type</returns> static public object GetBaseType(object o) { if (o is ArrayReturnType) { ArrayReturnType rt = o as ArrayReturnType; return rt.elementType; } return o; } }
Following table contains samples of variables with their Type
property.
N | Variable | Value of Type property |
Comment |
1 | double[] d = new double[3]; |
new ArrayReturnType((double)0, new int[]{3}, false); |
|
2 | object[] o = new object[2]; |
new ArrayReturnType((double)0, new int[]{2}, true); |
Every element of o is double |
3 | bool[,] b = new object[2,3]; |
new ArrayReturnType(false, new int[]{2, 3}, false); |
|
4 | string[,,] s = new string[5,2,3]; |
new ArrayReturnType("", new int[]{5, 2, 3}, false); |
Set of supported types of parameters can be extended.
Second interface is implemented by all providers of data.
/// <summary> /// Data provider /// </summary> public interface IMeasurements { /// <summary> /// The count of data units /// </summary> int Count { get; } /// <summary> /// Gets number - th unit of data /// </summary> IMeasure this[int number] { get; } /// <summary> /// Updates data /// </summary> void UpdateMeasurements(); /// <summary> /// Shows, weather the object is updated /// </summary> bool IsUpdated { get; set; } }
Following table explains meaning of this interface members
N | Member name | Meaning | Comment |
1 | Count |
The count of data units | |
2 | this[int number] |
Gets numberth unit of data | |
3 | UpdateMeasurements() |
Updates data | Data can be updated by the time |
4 | IsUpdated |
Shows, weather the object is updated | This property is used for avoiding multiple update |
All consumers of data implement following interface.
/// <summary> /// Consumer of data /// </summary> public interface IDataConsumer { /// <summary> /// Adds data provider /// </summary> /// <param name="measurements">Provider to add</param> void Add(IMeasurements measurements); /// <summary> /// Removes data provider /// </summary> /// <param name="measurements">Provider to remove</param> void Remove(IMeasurements measurements); /// <summary> /// Updates data of data providers /// </summary> void UpdateChildrenData(); /// <summary> /// Count of providers /// </summary> int Count { get; } /// <summary> /// Access to n - th provider /// </summary> IMeasurements this[int number] { get; } /// <summary> /// Resets measurements /// </summary> void Reset(); /// <summary> /// Change Input event /// </summary> event Action OnChangeInput; }
Following interface is auxiliary.
/// <summary> /// Collection on named data units /// </summary> public interface IAlias : IAliasBase { /// <summary> /// Names of all data units /// </summary> IList<string> AliasNames { get; } /// <summary> /// Access to data unit by name /// </summary> object this[string name] { get; set; } /// <summary> /// Gets unit type /// </summary> /// <param name="name">Unit name</param> /// <returns>Type of unit</returns> object GetType(string name); }
Following class links data consumers to data providers.
/// <summary> /// The link between data provider and data consumer /// </summary> [Serializable()] public class DataLink : ICategoryArrow, ISerializable, IRemovableObject, IDataLinkFactory { #region Fields /// <summary> /// Error message /// </summary> public static readonly string SetProviderBefore = "You should create measurements source before consumer"; /// <summary> /// DataLink checker /// </summary> private static Action<DataLink> checker; /// <summary> /// The source of this arrow /// </summary> private IDataConsumer source; /// <summary> /// The target of this arrow /// </summary> private IMeasurements target; /// <summary> /// Auxiliary field /// </summary> private int a = 0; /// <summary> /// Linked object /// </summary> protected object obj; /// <summary> /// Data link factory /// </summary> private static IDataLinkFactory dataLinkFactory = new DataLink(); #endregion #region Ctor /// <summary> /// Default constructor /// </summary> public DataLink() { } /// <summary> /// Deserialization constructor /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> public DataLink(SerializationInfo info, StreamingContext context) { a = (int)info.GetValue("A", typeof(int)); } #endregion #region ISerializable Members /// <summary> /// ISerializable interface implementation /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> public void GetObjectData(SerializationInfo info, StreamingContext context) { } #endregion #region ICategoryArrow Members /// <summary> /// The source of this arrow /// </summary> public ICategoryObject Source { set { if (source != null) { throw new Exception(); } IDataLinkFactory f = this; source = f.GetConsumer(value); } get { return source as ICategoryObject; } } /// <summary> /// The target of this arrow /// </summary> public ICategoryObject Target { get { return target as ICategoryObject; } set { if (target != null) { throw new Exception(); } IDataLinkFactory f = this; bool check = true; IAssociatedObject s = source as IAssociatedObject; if (s.Object != null & value.Object != null) { if (check) { INamedComponent ns = s.Object as INamedComponent; INamedComponent nt = value.Object as INamedComponent; if (nt != null & ns != null) { if (PureDesktopPeer.GetDifference(nt, ns) >= 0) { throw new Exception(SetProviderBefore); } } } target = t; source.Add(target); } if (!check) { return; } try { if (checker != null) { checker(this); } } catch (Exception e) { e.ShowError(10); source.Remove(target); throw e; } } } /// <summary> /// The "is monomorhpism" sign /// </summary> public bool IsMonomorphism { get { return false; } } /// <summary> /// The "is epimorhpism" sign /// </summary> public bool IsEpimorphism { get { return false; } } /// <summary> /// The "is isomorhpism" sign /// </summary> public bool IsIsomorphism { get { return false; } } /// <summary> /// Composes this arrow "f" with next arrow "g" /// </summary> /// <param name="category"> The category of arrow</param> /// <param name="next"> The next arrow "g" </param> /// <returns>Composition "fg" </returns> public ICategoryArrow Compose(ICategory category, ICategoryArrow next) { return null; } #endregion #region IAssociatedObject Members /// <summary> /// Associated object /// </summary> public object Object { get { return obj; } set { obj = value; } } #endregion #region IRemovableObject Members /// <summary> /// The post remove operation /// </summary> public void RemoveObject() { if (source == null | target == null) { return; } source.Remove(target); } #endregion #region IDataLinkFactory Members IDataConsumer IDataLinkFactory.GetConsumer(ICategoryObject source) { IAssociatedObject ao = source; object o = ao.Object; if (o is INamedComponent) { IDataConsumer dcl = null; INamedComponent comp = o as INamedComponent; IDesktop desktop = comp.Root.Desktop; desktop.ForEach<DataLink>(delegate(DataLink dl) { if (dcl != null) { return; } object dt = dl.Source; if (dt is IAssociatedObject) { IAssociatedObject aot = dt as IAssociatedObject; if (aot.Object == o) { dcl = dl.source as IDataConsumer; } } }); if (dcl != null) { return dcl; } } IDataConsumer dc = DataConsumerWrapper.Create(source); if (dc == null) { CategoryException.ThrowIllegalTargetException(); } return dc; } IMeasurements IDataLinkFactory.GetMeasurements(ICategoryObject target) { IAssociatedObject ao = target; object o = ao.Object; if (o is INamedComponent) { IMeasurements ml = null; INamedComponent comp = o as INamedComponent; IDesktop d = null; INamedComponent r = comp.Root; if (r != null) { d = r.Desktop; } else { d = comp.Desktop; } if (d != null) { d.ForEach<DataLink>(delegate(DataLink dl) { if (ml != null) { return; } object dt = dl.Target; if (dt is IAssociatedObject) { IAssociatedObject aot = dt as IAssociatedObject; if (aot.Object == o) { ml = dl.Target as IMeasurements; } } }); if (ml != null) { return ml; } } } IMeasurements m = MeasurementsWrapper.Create(target); if (m == null) { CategoryException.ThrowIllegalTargetException(); } return m; } #endregion #region Public Members /// <summary> /// Checker of data link /// </summary> public static Action<DataLink> Checker { set { checker = value; } } /// <summary> /// Data link factory /// </summary> public static IDataLinkFactory DataLinkFactory { get { return dataLinkFactory; } set { dataLinkFactory = value; } } /// <summary> /// Measurements provider /// </summary> public IMeasurements Measurements { get { return target; } } #endregion }
implies execution of following operator
consumer.Add(provider); // Consumer adds a provider.
Indeed above operators are never written explicitly, but they are implied by following graphical designer.
The Link arrow
corresponds to DataLink
object. Source
(Chart) (resp. target (Formula)) object is
object which implements IDataConsumer
(resp.
IMeasurements
) interface. Graphical setting of
Link arrow implies execution of next operator:
consumer.Add(provider); // Consumer adds a provider
Deleting of the arrow implies execution of next operator:
consumer.Remove(provider); // Consumer removes a provider
Following sample exhibits data exchange.
This sample contains three objects and three DataLink
arrows.
Following table contains types of objects.
N | Object | Type | Implemented interfaces |
1 | Formula | VectorFormulaConsumer |
IDataConsumer , IMeasurements ,
IAlias |
2 | Differential equation | DifferentialEquationSolver |
IDataConsumer , IMeasurements ,
IAlias |
3 | Chart | DataConsumer |
IDataConsumer |
VectorFormulaConsumer
and
DifferentialEquationSolver
implement both
IDataConsumer
and IMeasurements
interfaces. So these
objects can be both sources and targets of DataLink
arrow. But
DataConsumer
object can be source only. The
Formula object has following properties
Variables a and b are marked. It means that these variables
correspond to IAlias
interface. Following code snippet explains
this circumstance.
VectorFormulaConsumer formula = ...; IAlias alias = formula; IList<string> names = alias.AliasNames; // names[0] = "a", names[1] = "b"; double a = (double)alias["a"]; // a = 1; double b = (double)alias["b"]; // b = 2; object ta = alias.GetType("a"); // ta = (double)0; object tb = alias.GetType("b"); // tb = (double)0;
The t variable corresponds to virtual time.
The Differential equation object has following properties
This object performs solution of following system of ordinary differential equations
The Chart object indicates time dependencies. Its properties are clear.
Frame of reference is a basic notion of kinematics. Presented here architecture uses relative frames of reference, those have forest structure. Example of such structure is presented in the following diagram:
There exists a zero frame of reference. All root nodes of the forest correspond to this frame. 6D positions of other frames (nodes) are relative to 6D positions of those parents. It means that 6D position of frame 1.1.2 is relative to 6D position of frame 1.1 etc. This architecture enables us easily simulate a lot of different useful situations. For example we can install a set of virtual cameras on air(space)craft. Also it is easy to visualize complicated motion of helicopter rotor (slow motion) a plane with swing-wing or thrust vector control.
Following two interfaces correspond to 3D position and 3D orientation respectively
/// <summary> /// 3D Position /// </summary> public interface IPosition { /// <summary> /// Absolute position coordinates /// </summary> double[] Position { get; } /// <summary> /// Parent frame /// </summary> IReferenceFrame Parent { get; set; } /// <summary> /// Position parameters /// </summary> object Parameters { get; set; } /// <summary> /// Updates itself /// </summary> void Update(); }
/// <summary> /// Object with orientation /// </summary> public interface IOrientation { /// <summary> /// Orientation quaternion /// </summary> double[] Quaternion { get; } /// <summary> /// Orientation matrix /// </summary> double[,] Matrix { get; } }
The ReferenceFrame
class implements both IPosition
and IOrientation
. Its code is very long and is contained is source
files. Following interface supplies relative motion paradigm.
/// <summary> /// Reference frame holder /// </summary> public interface IReferenceFrame : IPosition { /// <summary> /// Own frame /// </summary> ReferenceFrame Own { get; } /// <summary> /// Children objects /// </summary> List<IPosition> Children { get; } }
The Parent
property of IReferenceFrame
interface is
the parent frame of reference. Motion of point is relative with respect to
parent frame. The Children
property of IReferenceFrame
is collection of children points. Following class links child to
parent.
/// <summary> /// Link of relative frame /// </summary> [Serializable()] public class ReferenceFrameArrow : CategoryArrow, ISerializable, IRemovableObject { #region Fields IPosition source; IReferenceFrame target; #endregion #region Constructors /// <summary> /// Default constructor /// </summary> public ReferenceFrameArrow() { } /// <summary> /// Deserialization constructor /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> protected ReferenceFrameArrow(SerializationInfo info, StreamingContext context) { } #endregion #region ISerializable Members void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context) { } #endregion #region ICategoryArrow Members /// <summary> /// The source of this arrow /// </summary> public override ICategoryObject Source { get { return source as ICategoryObject; } set { IPosition position = value.GetSource<IPosition>(); if (position.Parent != null) { throw new CategoryException("Root", this); } source = position; } } /// <summary> /// The target of this arrow /// </summary> public override ICategoryObject Target { get { return target as ICategoryObject; } set { IReferenceFrame rf = value.GetTarget<IReferenceFrame>(); IAssociatedObject sa = source as IAssociatedObject; IAssociatedObject ta = value as IAssociatedObject; INamedComponent ns = sa.Object as INamedComponent; INamedComponent nt = ta.Object as INamedComponent; target = rf; source.Parent = target; target.Children.Add(source); } } #endregion #region IRemovableObject Members void IRemovableObject.RemoveObject() { source.Parent = null; if (target != null) { target.Children.Remove(source); } } #endregion #region Specific Members /// <summary> /// Preparation operation /// </summary> /// <param name="collection">Desktop</param> /// <returns>List of position objects</returns> static public List<IPosition> Prepare(IComponentCollection collection) { List<IPosition> frames = new List<IPosition>(); if (collection == null) { return frames; } IEnumerable<object> c = collection.AllComponents; foreach (object o in c) { if (!(o is IObjectLabel)) { continue; } IObjectLabel lab = o as IObjectLabel; ICategoryObject co = lab.Object; if (!(co is IReferenceFrame)) { if (co is IPosition) { IPosition p = co as IPosition; if (p.Parent == null) { frames.Add(p); } } continue; } IReferenceFrame f = co as IReferenceFrame; if (f.Parent != null) { continue; } prepare(f, frames); } return frames; } /// <summary> /// Updates frames /// </summary> /// <param name="frames">List of frames</param> public static void Update(List<IPosition> frames) { foreach (IPosition frame in frames) { frame.Update(); } } private static void prepare(IReferenceFrame frame, List<IPosition> frames) { List<IPosition> children = frame.Children; frames.Add(frame); foreach (IPosition p in children) { if (frames.Contains(p)) { continue; } if (p is IReferenceFrame) { IReferenceFrame f = p as IReferenceFrame; prepare(f, frames); } else { frames.Add(p); } } } #endregion }
Source of this arrow is IPoint
object, target is a
IReferenceFrame
one. Setting of
ReferenceFrameArrow
arrow
IPosition position = ...; IReferenceFrame frame = ...; ReferenceFrameArrow link = new ReferenceFrameArrow(); link.Source = position as ICategoryObject; link.Target = position as ICategoryObject;
implies execution of following operators
position.Parent = frame; frame.Children.Add(position);.
Setting of all arrows is provided by graphical designer.
This picture has three links L1,
L2 and L3 of ReferenceFrameArrow
type and four objects
N | Name | Type | Implemented interfaces | Parent |
1 | Base frame | RigidReferenceFrame |
IPosition , IReferenceFrame |
null |
2 | Frame 1 | ReferenceFrameData |
IPosition , IReferenceFrame ,
IDataConsumer |
Base frame |
3 | Frame 2 | ReferenceFrameData |
IPosition , IReferenceFrame ,
IDataConsumer |
Frame 1 |
4 | Camera frame | RigidReferenceFrame |
IPosition , IReferenceFrame |
Frame 2 |
5 | Camera | WpfCamera |
IPosition |
Camera frame |
Since parent of Base frame is null
motion of
this frame is regarded as motion with respect to zero frame of reference. The
RigidReferenceFrame
class corresponds to frames with constant
relative coordinates and orientation. The ReferenceFrameData
class
corresponds to moving frames of reference. This class implements
IDataConsumer
interface for interoperability with data exchange.
The ReferenceFrameData
implements IDataConsumer
interface. It uses external data as parameters of relative motions. Following
picture represents application of ReferenceFrameData
The Motion Parameters object calculates motion parameters by following way:
This object as IMeasurements
is linked to Motion
Frame as IDataConsumer
by DataLink
D. The Motion Frame object uses output data of
Motion Parameters.
N | Motion parameter of Motion Frame | Output parameter of Motion Parameters |
1 | Relative x - coordinate | Formula_3 |
2 | Relative y - coordinate | Formula_4 |
3 | Relative z - coordinate | Formula_4 |
4 | Q0 component of relative orientation quaternion | Formula_1 |
5 | Q1 component of relative orientation quaternion | Formula_2 |
6 | Q2 component of relative orientation quaternion | Formula_4 |
7 | Q3 component of relative orientation quaternion | Formula_4 |
Besides IDataConsumer kinematic architecture contains
RelativeMeasurements
which implements IMeasurements
interface. Type of above Relative object is
RelativeMeasurements
. The R1, R2
arrow between Relative and frames mean that
Relative provides parameters of Motion Frame
relative motion with respect to Base frame. Since
Relative implements IMeasurements
interface it can
be connected by DataLink
to
IDataConsumer
Here Chart object indicates relative x, y and z coordinates.
Roughly speaking digital image processing obtains one image from other ones. So architecture contains provides and consumers of images. Following interface is provider of images.
/// <summary> /// Provider of image /// </summary> public interface IBitmapProvider { /// <summary> /// Bitmap /// </summary> Bitmap Bitmap { get; } }
Following interface is consumer of image.
/// <summary> /// Consumer of image /// </summary> public interface IBitmapConsumer { /// <summary> /// Procesess image /// </summary> void Process(); /// <summary> /// Providers /// </summary> IEnumerable<IBitmapProvider> Providers { get; } /// <summary> /// Adds a provider /// </summary> /// <param name="provider">The provider</param> void Add(IBitmapProvider provider); /// <summary> /// Removes a provider /// </summary> /// <param name="provider">The provider</param> void Remove(IBitmapProvider provider); }
Following class is link between
IBitmapProvider
and IBitmapConsumer
/// <summary> /// Link between bitmap consumer and bitmap provider /// </summary> [Serializable()] public class BitmapConsumerLink : ICategoryArrow, IRemovableObject, ISerializable { #region Fields /// <summary> /// Error message /// </summary> static public readonly string ProviderExists = "Bitmap provider already exists"; /// <summary> /// Error message /// </summary> public static readonly string SetProviderBefore = "You should create bitmap provider before consumer"; /// <summary> /// Associated object /// </summary> private object obj; /// <summary> /// Auxiliary variable /// </summary> private int a = 0; /// <summary> /// Source /// </summary> private IBitmapConsumer source; /// <summary> /// Target /// </summary> private IBitmapProvider target; #endregion #region Constructors public BitmapConsumerLink() { } public BitmapConsumerLink(SerializationInfo info, StreamingContext context) { info.GetValue("A", typeof(int)); } #endregion #region ICategoryArrow Members public ICategoryObject Source { get { return source as ICategoryObject; } set { source = value.GetObject<IBitmapConsumer>(); } } public ICategoryObject Target { get { return target as ICategoryObject; } set { target = value.GetObject<IBitmapProvider>(); source.Add(target); } } public bool IsMonomorphism { get { return false; } } public bool IsEpimorphism { get { return false; } } public bool IsIsomorphism { get { return false; } } public ICategoryArrow Compose(ICategory category, ICategoryArrow next) { return null; } #endregion #region IAssociatedObject Members public object Object { get { return obj; } set { obj = value; } } #endregion #region IRemovableObject Members public void RemoveObject() { if (source != null & target != null) { source.Remove(target); } } #endregion #region ISerializable Members public void GetObjectData(SerializationInfo info, StreamingContext context) { } #endregion #region Specific Members /// <summary> /// Updates consumer /// </summary> /// <param name="consumer">Consumer</param> public static void Update(IBitmapConsumer consumer) { IEnumerable<IBitmapProvider> providers = consumer.Providers; foreach (IBitmapProvider provider in providers) { if (provider is IBitmapConsumer) { IBitmapConsumer c = provider as IBitmapConsumer; Update(c); } } consumer.Process(); } /// <summary> /// Gets provider for consumer /// </summary> /// <param name="provider">Provider</param> /// <param name="consumer">Consumer</param> /// <param name="mutipleProviders">The multiple providers flag</param> /// <returns>The provider</returns> public static IBitmapProvider GetProvider(IBitmapProvider provider, IBitmapConsumer consumer, bool mutipleProviders) { if (provider == null) { return null; } ICategoryObject t = provider as ICategoryObject; ICategoryObject s = consumer as ICategoryObject; if (s.Object != null & t.Object != null) { INamedComponent ns = s.Object as INamedComponent; INamedComponent nt = t.Object as INamedComponent; if (nt != null & ns != null) { if (nt.Desktop == ns.Desktop) { if (nt.Ord >= ns.Ord) { throw new Exception(SetProviderBefore); } } else { if (nt.Root.Ord >= ns.Root.Ord) { throw new Exception(SetProviderBefore); } } } } return provider; } #endregion }
Setting of BitmapConsumerLink
arrow
IBitmapProvider provider = ...; IBitmapConsumer consumer = ...; BitmapConsumerLink link = new BitmapConsumerLink(); link.Source = provider as ICategoryObject; link.Target = consumer as ICategoryObject;
implies execution of following operator
consumer.Add(provider);. However above operations are performed by graphics designer.
Here Lady Rose and Lady Blue are
IBitmapProvider
and IBitmapConsumer
respectively.
These objects are linked by
BitmapConsumerLink
Digital image processing has classes which implement
IDataConsumer
and/or IMeasurements
interfaces. Let us
consider local digital filtration sample.
N | Object name | Type | Implemented interfaces |
1 | Earth | SourceBitmap |
IBitmapProvider |
2 | Result of processing | BitmapTransformer |
IBitmapProvider , IBitmapConsumer ,
IDataConsumer |
3 | Formulae | VectorFormulaConsumer |
IMeasurements , IAlias |
The Result of processing object as
IBitmapConsumer
is connected to Earth as
IBitmapProvider
by BitmapConsumerLink
. The
Result of processing object as IDataConsumer
is
connected to Formulae as IMeasurements
by
DataLink
. Any object of
SourceBitmap
class stores just stores image in memory. Main
members of this class are presented below.
#region Fields /// <summary> /// Bitmap /// </summary> protected Bitmap bitmap; #endregion #region Ctor /// <summary> /// Default constructor /// </summary> public SourceBitmap() { } /// <summary> /// Deserialization constructor /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> public SourceBitmap(SerializationInfo info, StreamingContext context) { try { bitmap = (Bitmap)info.GetValue("Bitmap", typeof(Bitmap)); } catch (Exception ex) { ex.ShowError(100); } } #endregion #region ISerializable Members /// <summary> /// ISerializable interface implementation /// </summary> /// <param name="info">Serialization info</param> /// <param name="context">Streaming context</param> public override void GetObjectData(SerializationInfo info, StreamingContext context) { info.AddValue("Bitmap", bitmap); } #region IBitmapProvider Members /// <summary> /// Bitmap /// </summary> Bitmap IBitmapProvider.Bitmap { get { return bitmap; } } #endregion /// <summary> /// Sets bitmap /// </summary> /// <param name="bitmap"></param> public void SetBitmap(Bitmap bitmap) { this.bitmap = bitmap; } // Other members ...
Business logic of SourceBitmap
is very clear. This class has a
field of Bitmap
type. User can set value of this field? This field
is serialized and is also used as Bitmap
of
IBitmapProvider
interface. The Formulae object has
following properties.
This formula calculates color from color of source bitmap. Parameters r, g and b correspond to red, green and blue color of source bitmap. If values of all colors exceed threshold value (a then formula returns x. Otherwise it returns y. Object Result of processing following properties.
These properties have following meaning. Object scans bitmap of provider object (Earth) and detects colors of pixels. It sets these objects as aliases of Formulae. Then it calculates Formulae and sets Formula_1 as red, green and blue color of result bitmap. Source bitmap and transformation result are presented below.
This algorithm is simplest rough algorithm of detection of snow mantle. C# explanation of this algorithm is presented below.
//============ Formulae ============== VectorFormulaConsumer formulae = ...; // Formulae object IMeasurements measurements = formulae; IMeasure m = measurements[0]; // Formula_1 IAlias alias = formulae; // Formulae object as IAlias // ========================================= // =========== Source bitmap =============== SourceBitmap sb = ...; IBitmapProvider provider = sb; Bitmap source = provider.Bitmap; //=========================================== // ============ Target bitmap =============== Bitmap target = // Size of tagtet is equal to source one new Bitmap(source.Width, source.Height); for (int x = 0; x < source.Width; x++) // x = coordinate of bitmap { for (int y = 0; y < source.Height; y++) // y - coordinate of bitmap { Color colorSource = source.GetPixel(x, y); // Color of source pixel double r = (double)colorSource.R / 256; // Scaling double g = (double)colorSource.G / 256; double b = (double)colorSource.B / 256; alias["r"] = r; // Setting of aliases alias["g"] = g; alias["b"] = b; measurements.UpdateMeasurements(); // Calculates all formulae double cd = (double)m.Parameter(); // Formula_1 int cdi = (int)(cd * 256); // Inverse scaling Color colorTarget = Color.FromArgb(cdi, cdi, cdi); // Target color target.SetPixel(x, y, colorTarget); // Sets pixel to target bitmap } }
Note that colors of target can be different as it is shown in "Lady Blue" sample. Above code snippet does not present in my code, it is just clear explanation of algorithm. More information about this subject you can find in my article "Digital Image Processing".
Several tasks imply simultaneous processing of several images. For example clouds dynamics indication requires comparison of two or more images. Following picture represents comparison of two images
This picture has following objects
N | Object name | Type | Implemented interfaces | Comment |
1 | Picture 1 | SourceBitmap |
IBitmapProvider |
First source image |
2 | Picture 2 | SourceBitmap |
IBitmapProvider |
Second source image |
3 | P 1 | BitmapColorTable |
IBitmapConsumer , IDataConsumer ,
IMeasurements |
Adapter object |
4 | P 2 | BitmapColorTable |
IBitmapConsumer , IDataConsumer ,
IMeasurements |
Adapter object |
5 | Input | VectorFormulaConsumer |
IMeasurements , IAlias |
Digital image processing calculator |
6 | Result | VectorFormulaConsumer |
IMeasurements , IAlias |
Digital image processing calculator |
7 | Compare | BitmapTransformer |
IBitmapProvider , IBitmapConsumer ,
IDataConsumer |
Digital image processing result |
The Compare object as IBitmapConsumer
is
connected to Picture 2 as IBitmapProvider
by
BitmapConsumerLink
. It means that
P 2 provides bitmap for Compare. According to
BitmapTransformer
implementation it means that size of
Compare bitmap is the same as Picture 2 one.
The Compare has following properties.
These properties have following meaning. The Compare object scans own bitmap and sets alias parameters as values of pixel coordinates. Then it sets Formula_1 value as red, green and blue color of image. Following code clarifies this algorithm.
VectorFormulaConsumer Input = ...; // The "Input" object VectorFormulaConsumer Result = ...; // The "Result" object IAlias alias = Input; // "Input" as IAlias IMeasurements measurements = // "Result" as IMeasurements Result; IMeasure Formula_1 = measurements[0]; // "Formula_1" Bitmap Compare = ...; // Bitmap of "Compare object" for (int x = 0; x < Compare.Width; x++) // x = coordinate of bitmap { for (int y = 0; y < Compare.Height; y++) // y - coordinate of bitmap { alias["x"] = (double)x; // Setting parameters alias["y"] = (double)y; measurements.UpdateMeasurements(); // Calculation int color = // Color (int)((double)Formula_1.Parameter() * 256); Color c = Color.FromArgb(color, color, color); Compare.SetPixel(x, y, c); // Sets pixel color } }
Both P 1 and P 2 are objects of
BitmapColorTable
type. This type as IDataConsumer
consumes coordinates pixel, and as IMeasurements
provides RGB color
parameters. Following picture
means that P 1 returns RGB parameters of Picture 1 bitmap pixel. Coordinate x (resp. y) of pixel equals to Formula_1, (resp. Formula_2) of Input. Properties of Input are presented below.
So parameter x (resp. y) of Input as
IAlias
equals to Formula_1 (resp. Formula_2) of
the same object as IMeasurements
. Thus input parameters of both
P 1, P 2 are pixel coordinates of
Compare bitmap. Properties of Result are
presented below.
Parameter x (resp. y) of this object is Red parameter of P 1 (resp. P 2), i.e. red component of pixel of Picture 1 (resp. Picture 2). Formula_1 of Result is proportional to difference between red components of Picture 1 and Picture 2.
Main interfaces of 3D graphics are IVisible
and
IVisibleConsumer
/// <summary> /// Object linked to position /// </summary> public interface IPositionObject { /// <summary> /// Linked position /// </summary> IPosition Position { get; set; } } /// <summary> /// Visible 3D object /// </summary> public interface IVisible : IPositionObject { } /// <summary> /// Consumer of visible 3D object /// </summary> public interface IVisibleConsumer { /// <summary> /// Adds visible object to consumer /// </summary> /// <param name="visible">Visible object to add</param> void Add(IVisible visible); /// <summary> /// Removes visible object from consumer /// </summary> /// <param name="visible">Visible object to remove</param> void Remove(IVisible visible); /// <summary> /// Post operation /// </summary> /// <param name="visible">Visible object</param> void Post(IVisible visible); }
Class VisibleConsumerLink
links
IVisibleConsumer
to IVisible
. Setting of
VisibleConsumerLink
arrow
IVisible visible = ...; IVisibleConsumer consumer = ...; ICategoryArrow link = new VisibleConsumerLink(); link.Source = consumer as ICategoryObject; link.Target = visible as ICategoryObject;
implies execution of following operator
consumer.Add(visible);
Removing of VisibleConsumerLink
follows
that
#region IRemovableObject Members /// <summary> /// Removing of VisibleConsumerLink /// </summary> void IRemovableObject.RemoveObject() { source.Remove(target); } #endregion
Following picture represents a sample of these interfaces.
Objects and arrows of above pictures have following types
N | Object name | Type | Implemented interfaces | Comment |
1 | Camera | WpfCamera |
IVisibleConsumer |
Wrapper of PerspectiveCamera |
2 | Globe | WpfShape |
IVisible |
Wrapper of Visual |
3 | Link | VisibleConsumerLink |
ICategoryArrow |
Links
IVisibleConsumer with IVisible object
|
Above example means that Camera consumes Globe, and "consumes" means that visualizes.
This architecture operates with interfaces which do not know about WPF. So this software can be adapted to other 3D graphics technologies, for example OpenGL. I have already described compatibility of this software with different 3D technologies in "The time machine" article.
Any object of 3D interface is associated with frame of reference. Next sample contains 3D object (Cube) with textures and four virtual cameras (Perspective Camera, X - camera, Y - camera and Z - camera).
The cube is linked by L 1 to moving frame of reference Frame 3 cameras are linked to fixed frames of reference. So cameras indicate Cube motion from different 3D points.
WPF implementation uses XAML file and texture files.
Following code is XAML file content
<ModelVisual3D xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"> <ModelVisual3D.Children> <ModelVisual3D> <ModelVisual3D.Content> <AmbientLight Color="#333333" /> </ModelVisual3D.Content> </ModelVisual3D> <ModelVisual3D> <ModelVisual3D.Content> <DirectionalLight Color="#FFFFFF" Direction="-0.612372,-0.5,-0.612372" /> </ModelVisual3D.Content> </ModelVisual3D> <ModelVisual3D> <ModelVisual3D.Content> <DirectionalLight Color="#FFFFFF" Direction="0.612372,-0.5,-0.612372" /> </ModelVisual3D.Content> </ModelVisual3D> <ModelVisual3D> <ModelVisual3D.Content> <GeometryModel3D> <GeometryModel3D.Material> <MaterialGroup> <DiffuseMaterial> <DiffuseMaterial.Brush> <ImageBrush Stretch="UniformToFill" ImageSource="leaves_closeup.png" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" /> </DiffuseMaterial.Brush> </DiffuseMaterial> <SpecularMaterial SpecularPower="85.3333"> <SpecularMaterial.Brush> <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/> </SpecularMaterial.Brush> </SpecularMaterial> </MaterialGroup> </GeometryModel3D.Material> <GeometryModel3D.Geometry> <MeshGeometry3D TriangleIndices="0,1,2 3,4,5" Normals="-1,0,0 -1,0,0 -1,0,0 -1,0,0 -1,0,0 -1,0,0 " TextureCoordinates="0,1 0,0 1,0 1,0 1,1 0,1 " Positions="-0.5,0.5,-0.5 -0.5,-0.5,-0.5 -0.5,-0.5,0.5 -0.5,-0.5,0.5 -0.5,0.5,0.5 -0.5,0.5,-0.5 " /> </GeometryModel3D.Geometry> </GeometryModel3D> </ModelVisual3D.Content> </ModelVisual3D> <ModelVisual3D> <ModelVisual3D.Content> <GeometryModel3D> <GeometryModel3D.Material> <MaterialGroup> <DiffuseMaterial> <DiffuseMaterial.Brush> <ImageBrush Stretch="UniformToFill" ImageSource="rocks.png" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" /> </DiffuseMaterial.Brush> </DiffuseMaterial> <SpecularMaterial SpecularPower="85.3333"> <SpecularMaterial.Brush> <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/> </SpecularMaterial.Brush> </SpecularMaterial> </MaterialGroup> </GeometryModel3D.Material> <GeometryModel3D.Geometry> <MeshGeometry3D TriangleIndices="0,1,2 3,4,5" Normals="0,0,1 0,0,1 0,0,1 0,0,1 0,0,1 0,0,1 " TextureCoordinates="0,0 1,0 1,1 1,1 0,1 0,0 " Positions="-0.5,-0.5,0.5 0.5,-0.5,0.5 0.5,0.5,0.5 0.5,0.5,0.5 -0.5,0.5,0.5 -0.5,-0.5,0.5 " /> </GeometryModel3D.Geometry> </GeometryModel3D> </ModelVisual3D.Content> </ModelVisual3D> <ModelVisual3D> <ModelVisual3D.Content> <GeometryModel3D> <GeometryModel3D.Material> <MaterialGroup> <DiffuseMaterial> <DiffuseMaterial.Brush> <ImageBrush Stretch="UniformToFill" ImageSource= "branches.png" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" /> </DiffuseMaterial.Brush> </DiffuseMaterial> <SpecularMaterial SpecularPower="85.3333"> <SpecularMaterial.Brush> <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/> </SpecularMaterial.Brush> </SpecularMaterial> </MaterialGroup> </GeometryModel3D.Material> <GeometryModel3D.Geometry> <MeshGeometry3D TriangleIndices="0,1,2 3,4,5" Normals="0,0,1 0,0,1 0,0,1 0,0,1 0,0,1 0,0,1 " TextureCoordinates="1,0 1,1 0,1 0,1 0,0 1,0 " Positions="0.5,-0.5,-0.5 0.5,0.5,-0.5 0.5,0.5,0.5 0.5,0.5,0.5 0.5,-0.5,0.5 0.5,-0.5,-0.5 " /> </GeometryModel3D.Geometry> </GeometryModel3D> </ModelVisual3D.Content> </ModelVisual3D> <ModelVisual3D> <ModelVisual3D.Content> <GeometryModel3D> <GeometryModel3D.Material> <MaterialGroup> <DiffuseMaterial> <DiffuseMaterial.Brush> <ImageBrush Stretch="UniformToFill" ImageSource="berries.jpg" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" /> </DiffuseMaterial.Brush> </DiffuseMaterial> <SpecularMaterial SpecularPower="85.3333"> <SpecularMaterial.Brush> <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/> </SpecularMaterial.Brush> </SpecularMaterial> </MaterialGroup> </GeometryModel3D.Material> <GeometryModel3D.Geometry> <MeshGeometry3D TriangleIndices="0,1,2 3,4,5" Normals="1,0,0 1,0,0 1,0,0 1,0,0 1,0,0 1,0,0 " TextureCoordinates="1,0 1,1 0,1 0,1 0,0 1,0 " Positions="-0.5,-0.5,-0.5 -0.5,0.5,-0.5 0.5,0.5,-0.5 0.5,0.5,-0.5 0.5,-0.5,-0.5 -0.5,-0.5,-0.5 " /> </GeometryModel3D.Geometry> </GeometryModel3D> </ModelVisual3D.Content> </ModelVisual3D> <ModelVisual3D> <ModelVisual3D.Content> <GeometryModel3D> <GeometryModel3D.Material> <MaterialGroup> <DiffuseMaterial> <DiffuseMaterial.Brush> <ImageBrush Stretch="UniformToFill" ImageSource="Waterlilies.png" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" /> </DiffuseMaterial.Brush> </DiffuseMaterial> <SpecularMaterial SpecularPower="85.3333"> <SpecularMaterial.Brush> <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/> </SpecularMaterial.Brush> </SpecularMaterial> </MaterialGroup> </GeometryModel3D.Material> <GeometryModel3D.Geometry> <MeshGeometry3D TriangleIndices="0,1,2 3,4,5 6,7,8 9,10,11" Normals="0,-1,0 0,-1,0 0,-1,0 0,-1,0 0,-1,0 0,-1,0 0,1,0 0,1,0 0,1,0 0,1,0 0,1,0 0,1,0 " TextureCoordinates="0,0 1,0 1,1 1,1 0,1 0,0 1,1 0,1 0,0 0,0 1,0 1,1 " Positions="-0.5,-0.5,-0.5 -0.5,0.5,0.5 -0.5,-0.5,0.5 0.5,-0.5,0.5 -0.5,-0.5,-0.5 -0.5,0.5,-0.5 0.5,0.5,-0.5 -0.5,0.5,-0.5 -0.5,0.5,0.5 -0.5,0.5,0.5 0.5,0.5,0.5 0.5,0.5,-0.5 " /> </GeometryModel3D.Geometry> </GeometryModel3D> </ModelVisual3D.Content> </ModelVisual3D> <ModelVisual3D > <ModelVisual3D.Content> <GeometryModel3D> <GeometryModel3D.Material> <MaterialGroup> <DiffuseMaterial> <DiffuseMaterial.Brush> <ImageBrush Stretch="UniformToFill" ImageSource="Sunset.jpg" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" /> </DiffuseMaterial.Brush> </DiffuseMaterial> <SpecularMaterial SpecularPower="85.3333"> <SpecularMaterial.Brush> <SolidColorBrush Color="#FFFFFF" Opacity="1.000000"/> </SpecularMaterial.Brush> </SpecularMaterial> </MaterialGroup> </GeometryModel3D.Material> <GeometryModel3D.Geometry> <MeshGeometry3D TriangleIndices="0,1,2 3,4,5 6,7,8 9,10,11" Normals="-1,0,0 -1,0,0 -1,0,0 -1,0,0 -1,0,0 -1,0,0 " TextureCoordinates="1,0 1,1 0,1 0,1 0,0 1,0 " Positions="-0.5,-0.5,0.5 -0.5,-0.5,-0.5 0.5,-0.5,-0.5 0.5,-0.5,-0.5 0.5,-0.5,0.5 -0.5,-0.5,0.5" /> </GeometryModel3D.Geometry> </GeometryModel3D> </ModelVisual3D.Content> </ModelVisual3D> </ModelVisual3D.Children> <ModelVisual3D.Transform> <Transform3DGroup > <Transform3DGroup.Children> <RotateTransform3D> <RotateTransform3D.Rotation> <AxisAngleRotation3D Angle="0" Axis="0 1 0" /> </RotateTransform3D.Rotation> </RotateTransform3D> <RotateTransform3D> <RotateTransform3D.Rotation> <AxisAngleRotation3D Angle="0" Axis="1 0 0" /> </RotateTransform3D.Rotation> </RotateTransform3D> <TranslateTransform3D OffsetX="0" OffsetY="0" OffsetZ="0" /> </Transform3DGroup.Children> </Transform3DGroup> </ModelVisual3D.Transform> </ModelVisual3D>
This content has references to texture files, for example
<ImageBrush Stretch="UniformToFill" ImageSource="berries.jpg" TileMode="None" ViewportUnits="Absolute" Viewport="0 0 1 1" AlignmentX="Left" AlignmentY="Top" Opacity="1.000000" />
WpfShape
class is a wrapper of these objects and it serializes
both XAML file and content of texture files.
info.AddValue("Xaml", xaml); info.AddValue("Textures", textures, typeof(Dictionary<string, byte[]>));
Following code explains how textures
dictionary is obtained.
string[] files = new string[] // File names { "berries.jpg", "branches.png", "leaves_closeup.png", "rocks.png", "Sunset.jpg", "Waterlilies.png" }; foreach (string fileName in files) { using (Stream stream = File.OpenRead(fileName)) { byte[] bytes = new byte[stream.Length]; stream.Read(bytes, 0, bytes.Length); // Reads file textures[fileName] = bytes; // Fills texture dictionary } }
Interoperability between digital image processing and 3D graphics is supported by replacing texture images with digital image processing ones. Following code represents essential features interoperability class.
/// <summary> /// 3D object with digital image processing interoperability /// </summary> [Serializable()] public class MotionImageFigure : WpfShape, IBitmapConsumer, IPostSetArrow { #region Fields /// <summary> /// Textures - Names of providers /// </summary> Dictionary<string, string> dTextures = new Dictionary<string, string>(); /// <summary> /// Textures - Bitmap poviders /// </summary> Dictionary<string, IBitmapProvider> providers = new Dictionary<string, IBitmapProvider>(); /// <summary> /// Post method /// </summary> protected virtual void Post() { foreach (string textureName in dTextures.Keys) // Textures cycle { string providerName = dTextures[textureName]; // Name of provider if (providers.ContainsKey(providerName)) { IBitmapProvider p = providers[providerName]; // Bitmap provider Bitmap bmp = p.Bitmap; // Bitmap of provider paths.Remove(textureName); using (MemoryStream stream = new MemoryStream()) { bmp.Save(stream, System.Drawing.Imaging.ImageFormat.Bmp); // Saving bitmap to bypes textures[textureName] = stream.GetBuffer(); // Replace texture by bitmap } } } }
Above Post()
method yields replacement of texture images by
digital image processing ones.
Following picture represents usage of MotionImageFigure
.
Type of Cube object is MotionImageFigure
. This
object has textures obtained from following graphical files:
Some of these textures can be replaced as.
Here textures are replaced by following way.
N | Texture file | Name of provider for replacement |
1 | rocks.png | Compare |
2 | branches.png | Result of processing |
In my "Determination of Orbits of Artificial Satellites" article a complicated engineering task is considered. Recently this task was extended by 3D animation and audio (see "Theory versus Practice?"). Virtual artificial satellite yields following animation picture.
Following files are used for above 3D animation.
Now we would like replace earth.png texture by digital image processing result. In result we have following.
Extract Containers.zip
directory to directory of
Aviation.exe
file.
Start Aviation.exe
.
Open OrbitImage.cfa
or OrbitTwoImage.cfa
.
Click following Animation button
Writing of this article inspires recollections. When I was a young engineer I worked with very bad documents. Long time later I worked with documents of US companies. These documents were related to different areas. However the documents are clear since they had uniform style. After that I try to unify style of my own documents.