Marginal Distributions of Continuous Random Variable

Transitive comparison of random variables

Bernard De Baets , ... Bart De Schuymer , in Logical, Algebraic, Analytic and Probabilistic Aspects of Triangular Norms, 2005

15.5.3 Continuous random variables

We first consider a continuous random vector ( X ₁, X ₂, …, X_m ) with arbitrary marginal distributions pairwisely coupled by T _M . We have demonstrated that [8]:

15.5.1 Proposition

Let (X ₁, X ₂, …, X_m ) be a continuous random vector. Then the probabilistic relation Q^M   =   [q^M _ij ] can be computed as:

(15.13) $q_{ij}^M={\int }_{x:{F_X}_{{}_i}\left(x\right)<F_{X_j}\left(x\right)}{f}_{X_i}\left(x\right)dx+\frac{1}{2}{\int }_{x:{F_X}_{{}_i}\left(x\right)<F_{X_j}\left(x\right)}{f}_{X_i}\left(x\right)dx\text{.}$

Note that if X_i and X_j are identically distributed, i.e. $F_{X_i}-F_{X_j}$ ,then $q_{ij}^M=\frac{1}{2}$ , as expected. In Figure 15.1 we give a graphical interpretation of formula(15.13). The two curves correspond to the marginal cumulative distribution functions F_X and F_Y .

Figure 15.1. Comparison of two continuous r.v. coupled by T _M .

According to (15.13) we have to distinguish between three domains: the domain where F_X lies beneath F_Y , the domain where F_X lies above F_Y , and the domain where F_X and F_Y coincide. The value of q^M _XY is computed as the sum of the increment of F_X over the first domain and half of the increment of F_X (or F_Y ) over the third domain. With the notations shown on the figure, we obtain for the example of Figure 15.1:

$q_{XY}^M=t_1+tl3+\frac{1}{2}{t}_2\text{.}$

This example also illustrates that the probabilistic relation associated to a random vector pairwisely coupled by T _M , can still be regarded as a graded version of the concept of stochastic dominance.

Note that the case of coupling by means of T _P , has been treated before as the case of independent random variables, and the computation of Q^P   =   [q^P _ij ] according to (15.9), can be concisely written as

$q_{ij}^P=E_{X_i}\left[F_{X_j}\right]\text{,}$

where $E_X\left[\cdots \right]$ denotes the expected value of the function between square brackets w.r.t. the distribution of X.

Next, we consider a continuous random vector (X ₁, X ₂, … ,X_m ) with arbitrary marginal distributions pairwisely coupled by T _L In [8], we have also shown that:

15.5.2 Proposition

Let (X ₁, X ₂, … ,X_m) be a continuous random vector. Then the probabilistic relation Q^L   =   [q^L _ij ] can be computed as:

$q_{ij}^L={\int }_{x:F_{X_i}\left(x\right)+F_{X_j}\left(x\right)\ge 1}{f}_{X_i}\left(x\right)dx\text{,}$

or, equivalently:

(15.14) $q_{ij}^L=F_{X_j}\left(u\right)\mathrm{with}u\mathrm{such}\mathrm{that}{F}_{X_i}\left(u\right)+F_{X_j}\left(u\right)=1\text{.}$

Note that u in (15.14) may not be unique, in which case any u fulfilling the right equality may be considered. Then q^L _ij is simply the height of $F_{X_j}$ in u. This is illustrated on Figure 15.2, where q^L _X , _Y   = F_Y (u)   = t ₁, since t ₁  + t ₂  =   1.

Figure 15.2. Comparison of two continuous r.v. coupled by T _L .

One can again easily verify that $Q_{XY}^L=\frac{1}{2}$ when F_X   = F_Y . Also, the probabilistic relation associated to a random vector pairwisely coupled by T _L , can again be regarded as a graded version of stochastic dominance.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B978044451814950015X

Multivariate stochastic orders

Félix Belzunce , ... Julio Mulero , in An Introduction to Stochastic Orders, 2016

3.5 The multivariate likelihood ratio order

In a similar way to the univariate case, it is possible to check the multivariate stochastic and hazard rate orders in terms of a property of the joint density functions. This property leads to the definition of the multivariate likelihood ratio order, which is a generalization of the univariate likelihood ratio order. In this section, the results are given in the continuous case but they can be restated for the general case. The main intention of the multivariate likelihood ratio order is to provide a sufficient condition for the multivariate hazard rate order.

Definition 3.5.1

Given two continuous random vectors X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) with joint density functions f and g, respectively, we say that X is smaller than Y in the multivariate likelihood ratio order, denoted by X ≤_lr Y, if

$f(\mathbf{x})g(\mathbf{y})\le f(\mathbf{x}\wedge \mathbf{y})g(\mathbf{x}\vee \mathbf{y}),\text{for all}\mathbf{x},\mathbf{y}\in {\mathbb{R}}^n.$

Clearly, this is a generalization of the likelihood ratio order in the univariate case. However, the multivariate likelihood ratio order is not necessarily reflexive. When a random vector X satisfies X ≤_lr X, we have the MTP₂ property introduced in Section 1.3.

The following result provides a set of sufficient conditions for the multivariate likelihood ratio order.

Theorem 3.5.2

Let X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) be two continuous random vectors with joint density functions f and g, respectively. If X or Y or both are MTP₂, and

(3.13) $\begin{array}{l}\hfill f(\mathbf{y})g(\mathbf{x})\le f(\mathbf{x})g(\mathbf{y}),\text{for all}\mathbf{x},\mathbf{y}\in {\mathbb{R}}^n,\text{such that}\mathbf{x}\le \mathbf{y},\end{array}$

then

$\mathbf{X}{\le }_{\text{lr}}\mathbf{Y}.$

Proof

Let us assume that X is MTP₂ (in the other case the proof is similar), then we have the following inequalities:

$f(\mathbf{x})g(\mathbf{y})\le \frac{f(\mathbf{x}\wedge \mathbf{y})f(\mathbf{x}\vee \mathbf{y})}{f(\mathbf{y})}g(\mathbf{y})\le f(\mathbf{x}\wedge \mathbf{y})g(\mathbf{x}\vee \mathbf{y}),$

for all $\mathbf{x},\mathbf{y}\in {\mathbb{R}}^n$ , such that x ≤y, where the first inequality follows from the MTP₂ property and the second one from (3.13). Therefore, X ≤_lr Y.

The multivariate likelihood ratio order is preserved under conditioning on sublattices, as we see next. Recall that a subset $A\subseteq {\mathbb{R}}^n$ is called a sublattice if x,y ∈ A implies x ∧y ∈ A and x ∨y ∈ A. This result will be used to show the relationship among the multivariate likelihood ratio order and the multivariate dynamic hazard rate order. The proof is obvious from the definition.

Theorem 3.5.3

Let X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) be two continuous random vectors. If X ≤_lr Y, then

$[\mathbf{X}|\mathbf{X}\subseteq A]{\le }_{\text{lr}}[\mathbf{Y}|\mathbf{Y}\subseteq A],\text{for all sublattice}A\subseteq {\mathbb{R}}^n.$

In particular, from the previous theorem, the multivariate likelihood ratio order is preserved under marginalization. This result is useful because, in some cases, it is easier to provide a result in the multivariate case rather that in the univariate case.

Theorem 3.5.4

Let X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) be two continuous random vectors. If X ≤_lr Y, then

${\mathbf{X}}_I{\le }_{\text{lr}}{\mathbf{Y}}_I,\text{for all}I\subseteq \{1,\dots ,n\}.$

Next, it is showed that the multivariate likelihood ratio order is stronger than the multivariate hazard rate order.

Theorem 3.5.5

Let X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) be two continuous random vectors. If X ≤_lr Y, then

$\mathbf{X}{\le }_{\text{hr}}\mathbf{Y}.$

Proof

Let us check the conditions for the definition of the multivariate dynamic hazard rate order. In particular, denoting by η and λ the multivariate dynamic hazard rates of X and Y, respectively, let us see if

${\eta }_k(t|h_t)\ge {\lambda }_k(t|h_t^{\prime }),\text{for all}t\ge 0,$

where

$h_t=\{{\mathbf{X}}_{I\cup J}={\mathbf{x}}_{I\cup J},{\mathbf{X}}_{\overline{I\cup J}}>t\mathbf{e}\},$

and

$h_t^{\prime }=\{{\mathbf{Y}}_I={\mathbf{y}}_I,{\mathbf{Y}}_{\overline{I}}>t\mathbf{e}\},$

whenever $I\cap J=\varnothing$ , 0 ≤x _I ≤y _I ≤ t e, 0 ≤x _J ≤ t e, and for all $k\in \overline{I\cup J}$ .

The result will follow by proving that, as we shall see later,

(3.14) $\begin{array}{l}\hfill \left[\left.{\mathbf{X}}_{\overline{I\cup J}}\right|h_t\right]{\le }_{\text{lr}}\left[\left.{\mathbf{Y}}_{\overline{I\cup J}}\right|h_t^{\prime }\right].\end{array}$

Condition (3.14) will follow if

(3.15) $\begin{array}{l}\hfill \left[\left.{\mathbf{X}}_{\overline{I\cup J}}\right|{\mathbf{X}}_{I\cup J}={\mathbf{x}}_{I\cup J}\right]{\le }_{\text{lr}}\left[\left.{\mathbf{Y}}_{\overline{I\cup J}}\right|{\mathbf{Y}}_I={\mathbf{y}}_I,{\mathbf{Y}}_J>t\mathbf{e}\right],\end{array}$

holds.

Let us see that (3.15) follows if X ≤_lr Y holds. Denoting by f and g the joint density functions of $({\mathbf{X}}_I,{\mathbf{X}}_J,{\mathbf{X}}_{I\cup J})$ and $({\mathbf{Y}}_I,{\mathbf{Y}}_J,{\mathbf{Y}}_{I\cup J})$ , respectively, and by f _I,J and g _I,J the joint densities of X _I,J and Y _I,J , respectively, we see that the joint density function of $\left[\left.{\mathbf{X}}_{\overline{I\cup J}}\right|{\mathbf{X}}_{I\cup J}={\mathbf{x}}_{I\cup J}\right]$ is given by

$f_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}})=\frac{f({\mathbf{x}}_I,{\mathbf{x}}_J,{\mathbf{x}}_{\overline{I\cup J}})}{f_{I,J}({\mathbf{x}}_I,{\mathbf{x}}_J)},$

and the joint density function of $\left[\left.{\mathbf{Y}}_{\overline{I\cup J}}\right|{\mathbf{Y}}_I={\mathbf{y}}_I,{\mathbf{Y}}_J>t\mathbf{e}\right]$ is given by

$g_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}})=\frac{{\int }_{{\mathbf{y}}_J>t\mathbf{e}}g({\mathbf{y}}_I,{\mathbf{y}}_J,{\mathbf{x}}_{\overline{I\cup J}})d{\mathbf{y}}_J}{{\int }_{{\mathbf{y}}_J>t\mathbf{e}}{g}_{I,J}({\mathbf{y}}_I,{\mathbf{y}}_J)d{\mathbf{y}}_J}.$

Given y _J > t e, we see that x _J < t e <y _J and, analogously, x _I ≤y _I . Since X ≤_lr Y, given ${\mathbf{y}}_{\overline{I\cup J}}$ , we see that

$\begin{array}{ll}\hfill & f({\mathbf{x}}_I,{\mathbf{x}}_J,{\mathbf{x}}_{\overline{I\cup J}})g({\mathbf{y}}_I,{\mathbf{y}}_J,{\mathbf{y}}_{\overline{I\cup J}})\hfill \\ \hfill & \le f({\mathbf{x}}_I,{\mathbf{x}}_J,{\mathbf{x}}_{\overline{I\cup J}}\wedge {\mathbf{y}}_{\overline{I\cup J}})g({\mathbf{y}}_I,{\mathbf{y}}_J,{\mathbf{x}}_{\overline{I\cup J}}\vee {\mathbf{y}}_{\overline{I\cup J}}),\hfill \end{array}$

which, upon integration, yields

$f_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}})g_{\overline{I\cup J}}({\mathbf{y}}_{\overline{I\cup J}})\le f_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}}\wedge {\mathbf{y}}_{\overline{I\cup J}})g_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}}\vee {\mathbf{y}}_{\overline{I\cup J}}).$

Therefore, from previous inequality, we see that (3.15) holds. Now, from Theorem 3.5.3, we see that (3.14) also holds. In particular, $\left[\left.X_k\right|h_t\right]{\le }_{\text{lr}}\left[\left.Y_k\right|h_t^{\prime }\right]$ holds, for all $k\in \overline{I\cup J}$ . Now, since (2.25), we see that the hazard rates of $\left[\left.X_k\right|h_t\right]$ and $\left[\left.Y_k\right|h_t^{\prime }\right]$ are ordered and, consequently, we see that

${\eta }_k(t|h_t)\ge {\lambda }_k(t|h_t^{\prime }).$

Finally, a result for the multivariate likelihood ratio order among random vectors with conditionally independent components is provided. In particular, we consider the same background as that in Theorem 3.3.8.

Theorem 3.5.6

Assume that

(i)

X _i (θ) =_st Y _i (θ), for all θ and for all i = 1,…,n,

(ii)

X _i (θ) ≤_lr Y _j (θ′), for all θ ≤ θ′ and for all 1 ≤ i ≤ j ≤ n, and

(iii)

θ ₁ ≤_lr θ ₂.

Then,

$(X_1,\dots ,X_n){\le }_{\text{lr}}(Y_1,\dots ,Y_n).$

Proof

Let f _i (x|θ) be the density function of X _i (θ). From condition (ii), we see that

$\prod _{i=1}^n{f}_i(x_i|{\theta }_i){\text{is TP}}_2\text{in}(x_1,\dots ,x_n,{\theta }_1,\dots ,{\theta }_n).$

Furthermore, condition (iii) is equivalent to the fact that h _i (θ ₁,…,θ _n ) is TP₂ in (θ ₁,…,θ _n ,i) ∈ S ×{1,2}. Due to these facts and from Theorem 1.2.3, we see that $\int f(x_1,\dots ,x_n|{\theta }_1,\dots ,{\theta }_n)d{H}_i(\mathit{\theta })$ is TP₂ in (x ₁,…,x _n ,i), and we get the result.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B978012803768300003X

Truth, Possibility and Probability

In North-Holland Mathematics Studies, 1991

Uniform distributions

Let a, b be real numbers with a < b. Then the standard density function

$f\left(x\right)=\{\begin{array}{cc}\frac{1}{b-a},& \text{if}a\le x\le b,\\ 0,& \text{otherwise}\cdot \end{array}$

is called a uniform density over [a, b]. It is clear that f is a density, since

${\int }_{-\infty }^{\infty }f\left(x\right)dx={\int }_a^{b}f\left(x\right)dx=1\cdot$

We now show a probability distribution that is approximately uniform on the interval [0, 1], i.e., with a uniform density over [0, 1]. We take the interval [0, 1] just for simplicity. Let Ω be an equally spaced hyperfinite approximation of the unit interval [0, 1]. That is, $\Omega =\{t_0,t_1,\dots ,t_v\}$ such that ν is an infinite natural number,t ₀ = 0,t _ν = 1, and we also asumme that $dt=t_{n+1}-t_n=1/v$ for every n ≤ ν. It is clear that #Ω = ν + 1. Thus, we are assuming that $\Omega =\{0,dt,2dt,3dt,\dots ,vdt\}$ . We define the distribution pr on Ω by setting for any n ≤ ν

$\text{pr}\left(t_n\right)=\frac{1}{\nu +1}\cdot$

Thus, for any internal subset A of Ω, we have the probability distribution

$PrA=\frac{\#A}{\#\Omega }\cdot$

The probability measure Pr is called the counting measure on Ω. It is clear that 〈Ω, pr〉 is an internal probability space. We now show that Pr has a uniform density.

The partition that we need in order to apply Theorem X.4 is formed by cubes, which in this case are intervals, $[t_0,t_1),\text{undefined}\dots ,\text{undefined}[t_i,t_{i+1}),\text{undefined}\dots ,\text{undefined}[t_{v-1},t_v)$ of length dt = 1/ν. Then

$\frac{Pr\left(\left[t_i,t_{i+1}\right)\cap \Omega \right)}{dt}=\frac{Pr\left(\left\{t_i\right\}\right)}{dt}=\frac{1}{\left(\nu +1\right)dt}\approx 1\cdot$

Since the uniform density for [0, 1] is f(x) = 1, for 0 ≤ x ≤ 1, by Theorem X.4, we get that Pr has this uniform density.

Thus, if c, d ∈ [0, 1] with c < d, we have

$Pr\left(\Omega \cap \left[c,d\right]\right)\approx {\int }_c^{d}dx=d-c\cdot$

We also can see that, in a sense, Theorem X.4, 2, cannot be improved: it is not true that for any partition with infinitesimal intervals, any interval r in the partition, and x ∈ r, satifies

$\frac{Pr\left(r\cap \Omega \right)}{V\left(r\right)}\approx 1\cdot$

This formula is true, as it is asserted in the theorem, only for partitions into cubes of volume ≥ 1/μ, for μ ≤ ν, for a certain fixed ν. In order to find a counterexample, take a partition with subintervals of length 1/2ν. Then there are some subintervals which do not contain any point in Ω, and, thus $Pr(r\cap \Omega )=0$ .

The same construction can be carried out in the unit circle C. As in Example VIII.4, let

$C=\left\{e^{ikd\theta }\Vert 0\le k\le \nu \right\},$

where $d\theta =2\pi /v$ and ν is an infinite natural number, be a hyperfinite approximation of C. Again, let pr be the counting measure on C. Then, as above, if A is an arc of C, we have that $Pr(C\cap A)\approx$ the length of A.

We discuss, now, Bertrand's mixture paradox (see page 61) from an infinitesimal standpoint. The uniform distribution of the ratio of water to wine between 1 and 2 is obtained by a near interval, $T_1=\{t_0,t_1,\dots ,t_v\}$ , for [1,2]. We must assume that $t_{i+1}-t_i=dt=1/(v+1)$ . Every point in T ₁ is equally probable, that is ${Pr}_1(t_i)=1/(v+1)$ .

On the other hand, the uniform distribution of the ratio of wine to water between 1/2 and 1, is obtained by an equally spaced near interval for [1/2,1]. If we take the reciprocals of the elements of T ₁, we do get a near interval, $T_2=\{u_0,u_1,\cdots ,u_v\}$ , for [1/2, 1], but its points are not equally spaced. Thus, we do not get in this way a uniform distribution. Suppose that we assign the same probability to each point, that is

${\text{pr}}_2\left(u_i\right)={\text{pr}}_1\left(t_i\right)=\frac{a}{\nu +1}.$

We have that

$d{u}_i=u_{i+1}-u_i=\frac{1}{t_{i+1}}-\frac{1}{t_i}=\frac{t_i-t_{i+1}}{t_i{t}_{i+1}}\cdot$

Then

$\begin{array}{ll}\frac{Pr\left([u_{i+1},u_i\right)\cap \Omega )}{d{u}_i}& =\frac{1}{\nu +1}\frac{t_i{t}_{i+1}}{t_{i+1}-t_i}\\ & =\frac{\nu }{\nu +1}\cdot t_i{t}_{i+1}\\ & \approx t_i^2\\ & =\frac{1}{u_i^2}\cdot \end{array}$

Thus, the density is, in this case, $f(x)=\frac{1}{x^2}$ , which is clearly not uniform.

We conclude this section with a corollary to the theorem on change of variables (Theorem IX.18). Notice, that if R is a region and T is continuously differentiate transformation defined on R, then T(R) is also a region.

Theorem X.7

Let $\overrightarrow{X}$ be a continuous random vector with density $f_{\overrightarrow{X}}$ over the region R, and let T be a continuously differentiable one-one transformation of R. Then the random vector $\overrightarrow{Y}=T(\overrightarrow{X})$ is also continuous with density

$f_{\overrightarrow{Y}}\left(\overrightarrow{y}\right)=f_{\overrightarrow{X}}\left(T^{-1}\left(\overrightarrow{y}\right)\right)\Vert J_{T^{-1}}\left(\overrightarrow{y}\right)\Vert ,$

for any $\overrightarrow{y}\in T(R)$ .

Proof

Let $\overrightarrow{X}$ be defined over 〈Ω, pr〉, and let R _ν and $P$ be as in (2) of Theorem X.7, such that $P$ is composed of infinitesimal cubes of area 1/μ. Thus, we have, for any r ∈ $P$ and $\overrightarrow{x}\in R$

$\begin{array}{cc}Pr\left[\overrightarrow{X}\in r\cap \Omega \right]={Pr}_{\overrightarrow{X}}\left(r\cap \Omega \right)\approx f\left(\overrightarrow{x}\right)V\left(r\right)& \left(V\left(r\right)\right)\end{array}\cdot$

Let $Q$ be the partition of T(R) obtained by applying T to the elements of $P$ . Then $Q$ is an infinitesimal partition of T(R), and, by Lemma IX.16

$\frac{V\left(T\left(r\right)\right)}{V\left(r\right)}\approx \Vert J_T\left(\overrightarrow{x}\right)\Vert \cdot$

We have for any s in $Q$ , since $f(T^{-1}(\overrightarrow{x}))$ and $J_T(\overrightarrow{x})$ are finite

$\begin{array}{ll}\frac{{Pr}_{\overrightarrow{Y}}\left(s\cap \Omega \right)}{V\left(s\right)}& =\frac{Pr\left[T\left(\overrightarrow{X}\right)\in s\right]}{V\left(s\right)}\\ & =\frac{Pr\left[\overrightarrow{X}\in T^{-1}\left(s\right)\right]}{V\left(s\right)}\\ & =\frac{Pr\left[\overrightarrow{X}\in T^{-1}\left(s\right)\right]}{V\left(T^{-1}\left(s\right)\right)}\frac{V\left(T^{-1}\left(s\right)\right)}{V\left(s\right)}\\ & \approx f\left(T^{-1}\left(\overrightarrow{x}\right)\right)\cdot \Vert J_T^{-1}\left(\overrightarrow{x}\right)\Vert \end{array}$

Since $f\circ T^{-1}\cdot \Vert J_{T^{-1}}\Vert$ is continuous and, by Theorem IX.18, integrable over T(R), by Theorem X.7 (1) it is a density of $\overrightarrow{Y}$ .

Recall that T is called an affine transformation of a region R if there exists an n × n matrix A and a vector $\overrightarrow{c}$ such that $T(\overrightarrow{x})=\overrightarrow{x}A+\overrightarrow{c}$ , for $\overrightarrow{x}\in R$ . If $\overrightarrow{c}=\overrightarrow{0}$ , then T is called linear. The function T is one-one, if A is nonsingular, and then

$T^{-1}\left(\overrightarrow{y}\right)=\left(\overrightarrow{u}-\overrightarrow{c}\right)A^{-1},$

for $\overrightarrow{y}\in T(R)$ . We have the following corollary.

Corollary X.8

Suppose that $\overrightarrow{X}$ is a continuous random vector with density $f_{\overrightarrow{X}}$ and that T is a one-one affine transformation, as defined above. Then $\overrightarrow{Y}=T(\overrightarrow{X})$ is continuous with density

$f_{\overrightarrow{Y}}\left(\overrightarrow{y}\right)=\Vert det{A}^{-1}\Vert f_{\overrightarrow{X}}\left(\left(\overrightarrow{y}-\overrightarrow{c}\right)A^{-1}\right),$

for every $\overrightarrow{y}{\in }^{n*}ℝ$ ,where det A is the determinant of A.

Proof

The corollary follows from the preceding theorem and

$J_T\left(T^{-1}\left(\overrightarrow{y}\right)\right)=detA\cdot$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/S0304020808724216

Lattice Vector Quantization for Wavelet-Based Image Coding

Mikhail Shnaider , Andrew P. Papliński , in Advances in Imaging and Electron Physics, 1999

B Distortion Measure and Quantization Regions

Let us consider an input stream of continuous random vectors x = (x ₁, x ₂, ..., x_N ) with the probability density function p( x ). Let us also consider an output stream y = (y ₁, y ₂, ..., y_N ) related to an input stream x by the conditional probability density function q(y|x ). Assuming that the stream x is the input to a quantizer, then the corresponding values from the output stream y are obtained by a reference to the quantizer codebook.

According to Shannon (1948), the amount of uncertainty R( x , y ) of a value of x when we receive its quantized counterpart y from a transmission channel is given by

(17) $R\left(x,y\right)=h\left(x\right)-h\left(x|y\right)$

where h( x ) = ∫p( x ) log p( x )dx is the differential entropy of a variable x The value of R( x , y ) is effectively the actual transmission rate. With this in mind, we can pose the problem of optimal quantization in terms of minimization of the amount of uncertainty, that is,

(18) $R\left(D\right)=\underset{q\in Q_D}{\min }\left(R\left(x,y\right)\right)$

with Q_D being a set specified by the conditional probability q( y | x ) so that

(19) $Q_D=\left\{q\left(y|x\right):E\left[d\left(x,y\right)\right]\le D\right\}$

where d(·,·) is adistance function (to be defined). The substitution of Eq. (17) in Eq. (18) leads to

(20) $R\left(D\right)=\underset{q\in Q_D}{\min }\left(h\left(x\right)-h\left(x|y\right)\right).$

As the entropy h( x ) is independent of the conditional probability q( y, x ) and the entropy h( x | y ) = h( x – x | y ) ≤ h( x − y ), we can further modify Eq. (20) in the following way:

(21) $\begin{array}{l}R(D)=h(x)-\underset{q\in Q_D}{\max }(h(x|y))\hfill \\ =h(x)-\underset{q\in Q_D}{\max }(h(x-y|y))\hfill \\ \ge h(x)-\underset{q\in Q_D}{\max }\left(h\left(x-y\right)\right).\hfill \end{array}$

In information theory the last equation is known as the Shannon lower bound(Berger, 1971; Sakrison, 1979; Gibson and Sayood, 1988).

Now let us return to the process of quantization. We collect the input vectors x _j = (x ₁, x ₂..., x _N) in blocks X = x ₁, x ₂..., x _L) and, for each input block, the quantizer finds a corresponding output block Y = y ₁, y ₂..., y _L consisting of vectors y_j = (y ₁, y ₂..., y _N). The average bit rate per vector for a vector quantizer can be calculated as

(22) $h_Q\left(Y\right)=\frac{1}{L}h\left(Y\right)=-\frac{1}{L}{\displaystyle \sum _{j}q\left(Y_j\right)\log q\left(Y_j\right).}$

Assuming that the distribution of x is uniform within each quantization region V and that all regions are simple translations of each other, we have the following approximation for the average bit rate per vector (Gibson and Sayood, 1988):

(23) $h_Q\left(Y\right)\approx \frac{1}{L}h\left(X\right)=-\frac{1}{L}\log V.$

If the input vectors x are independent we have that

$\begin{array}{ccc}h\left(x\right)=\frac{1}{L}h\left(X\right)& \text{and}& h\left(x-y\right)=\frac{1}{L}h\left(X-Y\right)\end{array}.$

Furthermore, it can be shown (Berger, 1971; Sakrison, 1979) that for sufficiently high dimensionality of vectors the following equality is satisfied:

(24) $h\left(X-Y\right)=\log V.$

Thus, we can conclude that the performance of an uniform vector quantizer asymptotically achieves the Shannon lower bound for large dimensions.

An average distortion of a quantizer can be defined in the following way:

(25) $D_Q^{\left(m\right)}={\displaystyle {\int }_{-\infty }^{\infty }{d}^{\left(m\right)}\left(x\right)p\left(x\right)dx}$

where for every input x with the PDF p(x) the quantizer produces an output y . The mth-power distortion d^(m) introduced by this operation is defined by

(26) $d_k^{{}^{\left(m\right)}}\left(x,y\right)={\left|y-x\right|}^m.$

By varying m, different distortion measures can be obtained. Closely related to the distortion measure is an optimal shape of the quantization region. It is easy to see that for a uniform source the optimal shape of quantization region is a pyramid, if m is selected to be 1, as depicted inFig. 11. For mequal to 2, Eq. (26) gives rise to the well-known mean squared error (MSE) distortion measure, the optimal quantization region for a uniform source being a sphere (Conway and Sloane, 1982). Furthermore, the radius of the sphere is equal to $\sqrt{LD}$ where D and L are the target distortion and dimensionality of quantization space, respectively.

Figure 11. A pyramidal quantization region for m = 1.

From the foregoing considerations we can conjecture that, assuming the MSE distortion measure, an optimal quantizer for an L-dimensional uniform source constitutes a set of uniformly distributed spheres with code vectors as centroids of the spheres.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/S1076567008701988

Reliability Theory

N. Unnikrishnan Nair , ... N. Balakrishnan , in Reliability Modelling and Analysis in Discrete Time, 2018

Blomqvist's β

The Blomqvist measure is also based on concordance, but with the difference that instead of two random vectors here we choose a random vector and a fixed point $(x_0,y_0)$ with $x_0$ as the median of $X_1$ and $y_0$ as the median of $X_2$ . Thus, the measure in the case of a continuous random vector $(X_1,X_2)$ becomes

$\begin{array}{cc}\hfill \beta & =P[(X_1-x_0)(X_2-y_0)>0]-P[(X_1-x_0)(X_2-y_0)<0]\hfill \\ \hfill & =2\{P[X_1<x_0,X_2<y_0]+P[X_1>x_0,X_2>y_0]\}-1.\hfill \end{array}$

In the discrete case, this modifies to

(1.67) $\beta =4P[X_1<x_0,X_2<y_0]-1+P[X_1=x_0\text{ or}{X}_2=y_0].$

The medians are chosen to be the nearest integer values.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128019139000014

Preliminaries

Félix Belzunce , ... Julio Mulero , in An Introduction to Stochastic Orders, 2016

1.3.1 The standard construction

The problem of defining a suitable multivariate extension of the univariate quantile function has a long history in statistics and probability. Next, the definition of the multivariate quantile transform is recalled. This transformation is also known as the standard construction, and is introduced by Arjas and Lehtonen [28], O'Brien [29], Rosenblatt [30], or Rüschendorf [31 ]. Given a continuous random vector X = (X ₁,…,X _n ), its standard construction is recursively defined as

(1.11) $\begin{array}{c}\hfill \begin{array}{c}{Q}_{\mathbf{X},1}(p_1)=F_{X_1}^{-1}(p_1),\\ \\ Q_{\mathbf{X},2}(p_1,p_2)=F_{[X_2|X_1=Q_{\mathbf{X},1}(p_1)]}^{-1}(p_2),\\ ⋮\\ Q_{\mathbf{X},n}(p_1,\dots ,p_n)=F_{\left[X_n\left|{\cap }_{j=1}^{n-1}\left\{X_j=Q_{\mathbf{X},j}(p_1,\dots ,p_j)\right\}\right.\right]}^{-1}(p_n),\end{array}\hfill \end{array}$

for all (p ₁,p ₂,…,p _n ) ∈ (0,1) ⁿ , where $F_{X_1}^{-1}$ denotes the quantile function of X ₁ and

$F_{\left[X_i|{\bigcap }_{j=1}^{i-1}\left\{X_j=Q_{\mathbf{X},j}(p_1,\dots ,p_j)\right\}\right]}^{-1},\text{for all}i=2,\dots ,n,$

denotes the quantile functions of the univariate conditional random variables given by

$\left[X_i\left|\bigcap _{j=1}^{i-1}\left\{X_j=Q_{\mathbf{X},j}(p_1,\dots ,p_j)\right\}\right.\right],\text{for all}i=2,\dots ,n.$

It is worth mentioning that this well-known transform is widely used in simulation theory and it plays the role of the quantile in the multivariate case. Hence, given n independent uniformly distributed random variables U ₁,…,U _n on the interval (0,1), then

(1.12) $\begin{array}{l}\hfill (X_1,\dots ,X_n){=}_{\text{st}}{Q}_X(U_1,\dots ,U_n),\end{array}$

where

$Q_X(p_1,\dots ,p_n)=(Q_{\mathbf{X},1}(p_1),Q_{\mathbf{X},2}(p_1,p_2),\dots ,Q_{\mathbf{X},n}(p_1,\dots ,p_n)),$

for all (p ₁,…,p _n ) ∈ (0,1) ⁿ . As we shall see later, another transform of interest is the multivariate distributional transform. Given a continuous random vector X = (X ₁,…,X _n ), its multivariate distributional transform is recursively defined as

(1.13) $\begin{array}{c}\hfill \begin{array}{c}{D}_{\mathbf{X},1}(x_1)=F_{X_1}(x_1),\\ \\ D_{\mathbf{X},2}(x_1,x_2)=F_{[X_2|X_1=x_1]}(x_2),\\ ⋮\\ D_{\mathbf{X},n}(x_1,\dots ,x_n)=F_{\left[X_n\left|{\cap }_{j=1}^{n-1}\left\{X_j=x_j\right\}\right.\right]}(x_n),\end{array}\end{array}$

for all (x ₁,…,x _n ) in the support of X, where $F_{X_1}$ denotes the distribution function of X ₁ and

$F_{\left[X_i\left|\right.{\bigcap }_{j=1}^{i-1}\left\{X_j=x_j\right\}\right]},\text{for all}i=2,\dots ,n$

denote the distribution function of the conditional random variables given by

$\left[X_i\left|\bigcap _{j=1}^{i-1}\left\{X_j=x_j\right\}\right.\right],\text{for all}i=2,\dots ,n.$

According to the previous notation, we find that

(1.14) $\begin{array}{l}\hfill (U_1,\dots ,U_n){=}_{\text{st}}{D}_X(X_1,\dots ,X_n),\end{array}$

where

$D_X(x_1,\dots ,x_n)=(D_{\mathbf{X},1}(x_1),D_{\mathbf{X},2}(x_1,x_2),\dots ,D_{\mathbf{X},n}(x_1,\dots ,x_n)),$

for all $(x_1,\dots ,x_n)\in {\mathbb{R}}^n$ . Given two continuous random vectors X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ), we can consider the transformation

(1.15) $\begin{array}{l}\hfill \mathbf{\Phi }(x_1,\dots ,x_n)=Q_Y(D_X(x_1,\dots ,x_n)),\end{array}$

defined for all (x ₁,…,x _n ) in the support of X. Observe that

$\begin{array}{c}{\mathbf{\Phi }}_1(x_1)=F_{Y_1}^{-1}(F_{X_1}(x_1)),\\ \\ {\mathbf{\Phi }}_2(x_1,x_2)=F_{\left[Y_2\left|Y_1={\mathbf{\Phi }}_1(x_1)\right.\right]}^{-1}\left(F_{[X_2|X_1=x_1]}(x_2)\right),\\ ⋮\\ {\mathbf{\Phi }}_n(x_1,\dots ,x_n)=F_{\left[Y_n\left|{\cap }_{j=1}^{n-1}\left\{Y_j={\mathbf{\Phi }}_j(x_1,\dots ,x_j)\right\}\right.\right]}^{-1}\left(F_{\left[X_n\left|{\displaystyle {\cap }_{j=1}^{n-1}\{X_j=x_j\}}\right.\right]}(x_n)\right),\end{array}$

for all (x ₁,…,x _n ) in the support of X. Since the distribution and quantile functions are increasing, then Φ _i (x ₁,…,x _i ) is also increasing in x _i , for all i = 1,…,n. In fact, in cases of differentiability, the Jacobian matrix of Φ is always a lower triangular matrix with strictly positive diagonal elements.

The most relevant property of Φ is that, from (1.12) and (1.14), we have

$\mathbf{Y}{=}_{\text{st}}\mathbf{\Phi }(\mathbf{X}),$

and, consequently, the function Φ maps the random vector X onto Y.

In addition, Fernández-Ponce and Suárez-Llorens [32] prove in their Theorem 3.1 that if we take a function $\mathbf{k}:{\mathbb{R}}^n\mapsto {\mathbb{R}}^n$ such that Y =_st k(X) and k has a lower triangular Jacobian matrix with strictly positive diagonal elements, then k necessarily has the form of the function Φ given in (1.15).

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128037683000016

Fundamental of predictive filtering

Lu Cao , ... Bing Xiao , in Predictive Filtering for Microsatellite Control System, 2021

2.2.1 Random vector

In Probability Theory, a random vector is defined as $\mathit{X}={[X_1{X}_2\cdots X_n]}^{\mathrm{T}}$ , where $X_i$ is a random variable, $i=1,2,\cdots ,n$ . Its joint cumulative distribution function is defined as the probability of an n-dimensional semi-infinite rectangle associated with the sample point $\mathit{x}={[x_1{x}_2\cdots x_n]}^{\mathrm{T}}$ . It is given by

(2.1) ${\mathit{F}}_{\mathit{X}}(\mathit{x})={\mathit{F}}_{\mathit{X}}(x_1,x_2,\cdots ,x_n)=\mathbb{P}(X_1⩽x_1,X_2⩽x_2,\cdots ,X_n⩽x_n)$

If X is continuous random vector, then its joint probability density function is defined as

(2.2) ${\mathit{f}}_{\mathit{X}}(\mathit{x})=\frac{{\partial }^n}{\partial x_1\cdots \partial x_n}{\mathit{F}}_{\mathit{X}}(x_1,x_2,\cdots ,x_n)$

For a discrete vector X , its joint probability mass function is defined as

(2.3) ${\mathcal{P}}_{\mathit{X}}(\mathit{x})=\mathbb{P}(X_1=x_1,X_2=x_2,\cdots ,X_n=x_n)$

The joint probability density function and the joint probability mass function satisfy

(2.4) ${\int }_{-\infty }^{\infty }\cdots {\int }_{-\infty }^{\infty }{\mathit{f}}_{\mathit{X}}(\mathit{x})d{x}_1\cdots d{x}_n=1$

(2.5) $\sum _{x_1}\cdots \sum _{x_n}{\mathcal{P}}_{\mathit{X}}(\mathit{x})=1$

Moreover, for all continuous random vectors, ${\mathit{f}}_{\mathit{X}}(\mathit{x})$ is such that

(2.6) ${\mathit{F}}_{\mathit{X}}(\mathit{x})={\int }_{-\infty }^{x_1}{\int }_{-\infty }^{x_2}\cdots {\int }_{-\infty }^{x_n}{\mathit{f}}_{\mathit{X}}({\tau }_1,{\tau }_2,\cdots ,{\tau }_n)d{\tau }_{1}d{\tau }_2\cdots d{\tau }_n$

Let $\mathit{X}={[X_1{X}_2\cdots X_n]}^{\mathrm{T}}$ and $\mathit{Y}={[Y_1{Y}_2\cdots Y_m]}^{\mathrm{T}}$ be two random vectors having n and m components, respectively. The joint cumulative distribution function of X and Y at $\mathit{x}={[x_1{x}_2\cdots x_n]}^{\mathrm{T}}$ and $\mathit{y}={[y_1{y}_2\cdots y_m]}^{\mathrm{T}}$ is defined as

(2.7) ${\mathit{F}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})=\mathbb{P}(X_1⩽x_1,\cdots ,X_n⩽x_n,Y_1⩽y_1,\cdots ,Y_m⩽y_m)$

When X and Y are discrete, we define the joint probability mass function of X and Y as

(2.8) ${\mathcal{P}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})=\mathbb{P}(X_1=x_1,\cdots ,X_n=x_n,Y_1=y_1,\cdots ,Y_m=y_m)$

and their marginal probability functions as

(2.9) ${\mathcal{P}}_{\mathit{X},\mathit{X}}(\mathit{x})=\sum _{y_1}\cdots \sum _{y_m}{\mathcal{P}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})$

(2.10) ${\mathcal{P}}_{\mathit{Y},\mathit{Y}}(\mathit{y})=\sum _{x_1}\cdots \sum _{x_n}{\mathcal{P}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})$

For continuous X and Y , we define their joint probability density function as

(2.11) ${\mathit{f}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})=\frac{{\partial }^{n+m}}{\partial x_1\cdots \partial x_n\partial y_1\cdots \partial y_m}{\mathit{F}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})$

and their marginal probability functions as

(2.12) ${\mathit{f}}_{\mathit{X},\mathit{X}}(\mathit{x})={\int }_{-\infty }^{\infty }\cdots {\int }_{-\infty }^{\infty }{\mathit{f}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})d{y}_{1}d{y}_2\cdots d{y}_m$

(2.13) ${\mathit{f}}_{\mathit{Y},\mathit{Y}}(\mathit{y})={\int }_{-\infty }^{\infty }\cdots {\int }_{-\infty }^{\infty }{\mathit{f}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})d{x}_{1}d{x}_2\cdots d{x}_n$

Then, the random vectors X and Y are independent if and only if

(2.14) $\{\begin{array}{cc}\hfill {\mathcal{P}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})& ={\mathcal{P}}_{\mathit{X},\mathit{X}}(\mathit{x}){\mathcal{P}}_{\mathit{Y},\mathit{Y}}(\mathit{y}),\text{if}\mathit{X}\text{ and}\mathit{Y}\text{undefined}\text{are discrete}\hfill \\ \hfill {\mathit{f}}_{\mathit{X},\mathit{Y}}(\mathit{x},\mathit{y})& ={\mathit{f}}_{\mathit{X},\mathit{X}}(\mathit{x}){\mathit{f}}_{\mathit{Y},\mathit{Y}}(\mathit{y}),\text{if}\mathit{X}\text{ and}\mathit{Y}\text{undefined}\text{are continuous}\hfill \end{array}$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128218655000152

The Analysis of Structural Equation Model with Ranking Data using Mx

Wai-Yin Poon , in Handbook of Latent Variable and Related Models, 2007

1 Introduction

Ranking data are obtained when subjects are asked to rank p objects from 1 (most preferred) to p (least preferred). A number of approaches have been developed to model and analyze ranking data. Among others, researches on using Thurstonian models (Thurstone, 1927) remain active for more than 70 years (see, e.g., Critchlow and Fligner, 1991; Böckenholt, 1992, 1996, 2001; Chan and Bentler, 1998; Mayedu-Olivares, 1999), confirming their usefulness in a wide range of disciplines. Thurstonian models postulate that the ranking of the p objects are determined by a p × 1 latent continuous random vector Y which is distributed as multivariate normal with mean μ and covariance matrix Σ. Different models are achieved by putting different restrictions on the elements in Σ. In particular, the use of structural equation models by imposing the structures on Σ substantially enriches the horizon of modeling ranking data and hence its practical applicability (see, e.g., Currim, 1982; Böckenholt, 1992; Elrod and Keane, 1995).

In effect, structural equation modeling remains an extremely active area of research. The interest derives from both the practical applicability of the approach in addressing and verifying substantive theories and the technical difficulties involving in modeling of various types of data together with the estimation of the resulted model. User-friendly structural equation modeling packages are widely available, such as PRELIS and LISREL (Jöreskog and Sörbom, 1996a, 1996b), EQS (Bentler and Wu, 1993), Mx (Neale et al., 1999) as well as Mplus (Muthén and Muthén, 1998). The capabilities of these packages continue to be extended and nowadays, they all provide options for analyzing ordinal categorical data in a convenient manner. Although ranking data and ordinal categorical data are different in nature and require different modeling and estimation techniques, the approaches adopted by popular packages for analyzing ordinal categorical data and the Thurstonian approach for analyzing ranking data both operate on the assumption that the observed variables are associated with some underlying continuous variables distributed as multivariate normal. With reference to this similarity, the current study establishes a relationship between the analysis of ordinal categorical data and ranking data with a view to making use of readily available structural equation modeling procedures for analyzing ordinal categorical data to analyze ranking data. Specifically, the implementation of the analysis of the Thurstonian models using options designated for analyzing ordinal categorical data in the widely available software program Mx (Neale et al., 1999) is addressed. Some initial effort along similar direction has been made by Mayedu-Olivares (1999). He has developed a procedure for analyzing ranking data using the package MECOSA (Arminger et al., 1996).

A description of the Thurstonian model of ranking data, the multivariate normal model for analyzing ordinal categorical data, and their similarity is given in Section 2. A summary of the Mx program together with its option for analyzing ordinal categorical data, and the procedure on using the option to implement the Thurstonian models are given in Section 3. It will be seen that the procedure is very flexible. Different identification and across parameters constraints together with various structures on the mean vector and on the covariance/correlation matrix can be incorporated in an efficient and easy manner. Some examples available in the literature are used for illustration. In effect, the procedure can also be applied to analyze models for partial ranking data (Böckenholt, 1992). Two applications are discussed in Section 4. The first application relates to an analysis of rankings of compound objects. The objective is to examine whether or not a mean score for a compound choice alternative consisting of two objects can be predicted by an additive combination of means scores obtained for each of the two objects separately. The second application relates to the analysis of a set of 8 soft drinks, and the rankings of the soft drinks are obtained via a balanced incomplete design. The paper is concluded with a discussion in Section 5.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780444520449500124

Elicitation of multivariate prior distributions: A nonparametric Bayesian approach

Fernando A. Moala Anthony O'Hagan , in Journal of Statistical Planning and Inference, 2010

We assume that the expert's prior density is a relatively smooth function $f(·):{\mathfrak{R}}^k\to \mathfrak{R}$ of an unknown continuous random vector $\mathit{\theta }=({\theta }_1,{\theta }_2,\dots ,{\theta }_k)$ . We will model a nonparametric prior for $f(·)$ hierarchically in terms of a vector $\mathit{\alpha }$ of hyperparameters. An appropriate choice to represent the analyst's prior beliefs about $f(·)$ is a Gaussian process, which we denote by

(1) $f(·)|\mathit{\alpha }\sim \mathit{GP}(g(·),{\sigma }^{2}C(·,·))\text{,}$

where the mean and covariance functions

(2) $E\{f(\mathit{\theta })|\mathit{\alpha }\}=g(\mathit{\theta }),\mathit{cov}[f(\mathit{\theta }),f(\mathit{\varphi })|\mathit{\alpha }]={\sigma }^{2}C(\mathit{\theta },\mathit{\varphi })$

depend on the hyperparameters $\mathit{\alpha }$ .

Read full article

URL:

https://www.sciencedirect.com/science/article/pii/S0378375810000157

petersonsomuckledge.blogspot.com

Source: https://www.sciencedirect.com/topics/mathematics/continuous-random-vector

Share this post