Does mean centering reduces covariance?












2












$begingroup$


It might be a very naive question, but assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal". Does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so thinking it should do the same for covariance.






correlation covariance random-vector







share|cite|improve this question









$endgroup$

















    2












    $begingroup$


    It might be a very naive question, but assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal". Does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so thinking it should do the same for covariance.






    correlation covariance random-vector







    share|cite|improve this question









    $endgroup$















      2












      2








      2


      1



      $begingroup$


      It might be a very naive question, but assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal". Does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so thinking it should do the same for covariance.






      correlation covariance random-vector







      share|cite|improve this question









      $endgroup$




      It might be a very naive question, but assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal". Does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so thinking it should do the same for covariance.






      correlation covariance random-vector




      correlation covariance random-vector






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked 2 hours ago









      lvdplvdp

      113




      113






















          1 Answer
          1






          active

          oldest

          votes


















          3












          $begingroup$

          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "65"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f391838%2fdoes-mean-centering-reduces-covariance%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          3












          $begingroup$

          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago
















          3












          $begingroup$

          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago














          3












          3








          3





          $begingroup$

          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.






          share|cite|improve this answer











          $endgroup$



          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.







          share|cite|improve this answer














          share|cite|improve this answer



          share|cite|improve this answer








          edited 1 hour ago

























          answered 1 hour ago









          Artem MavrinArtem Mavrin

          34516




          34516












          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago


















          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago
















          $begingroup$
          Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
          $endgroup$
          – lvdp
          26 secs ago




          $begingroup$
          Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
          $endgroup$
          – lvdp
          26 secs ago


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Cross Validated!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f391838%2fdoes-mean-centering-reduces-covariance%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          Statuo de Libereco

          Image
          Koordinatoj: 40°41′21″N 74°02′40″W   /   40.68917°N , 74.04444°U  / 40.68917; -74.04444  ( Statuo de Libereco ) Statuo de Libereco angle: Statue of Liberty statuo Statuo de Libereco de antaŭe Oficiala nomo: Liberty Enlightening the World Lando  Usono Ŝtato Novjorkio Regiono Novjorko Insulo Insulo de Libereco Situo Statuo de Libereco  - koordinatoj 40°41′21″N 74°02′40″W   /   40.68917°N , 74.04444°U  / 40.68917; -74.04444  ( Statuo de Libereco ) Alteco de statuo 46 m  - alteco kun soklo 93 m Pezo 205 000 kg (451 948 lb) Materialoj bronzo, ŝtalo Proponinto Gustave Eiffel  - skulptisto Frédéric Auguste Bartholdi  - proponinto de soklo Richard Morris Hunt  - kapo laŭ modelo de Isabella Eugenie Boyer Transdonita 4-an de julio 1886 Por publiko alirebla por publiko Horzono OT (UTC-5)  - somera tempo OT (UTC-4) Loko
          Read more

          Tanganjiko

          Image
          Lago Tanganjiko Situo centra Afriko Ĉefaj fontoj Kalambo, Lufuko, Malagarasi kaj Ruzizi Ĉefaj elfluoj Lukugo Setlejoj Bujumbura, Kalemie Datenoj Koordinatoj 6°  S , 30°  O -6.3827777777778 29.615 782 Koordinatoj: 6°  S , 30°  O DEC Supermara alteco 782 m Surfaca areo 32 893 km² f5 Maksimuma longo 673 km f6 Maksimuma larĝo 72 km f7 Akva volumeno 18.900 km³ dep1 f8 Maksimuma profundo 1470 m f10 Averaĝa profundo 570 m f11 v   •   d   •   r Mapo de la lago Tanganjiko estas lago en orienta Afriko, inter Burundo, Kongo Kinŝasa, Tanzanio kaj Zambio. Ĝi apartenas al la Granda Rifto kaj estas unu el la Afrikaj Grandaj Lagoj. Tanzanio (46%) kaj DRK (40%) havas la majoritaton de la lago. Ĝi havas longon de 673 km, areon de ĉ. 32.900 km² kaj maksimuman profundon taksatan je 1470 metroj; pro tiu ĉi granda profundo ankaŭ ĝia volumeno estas sufiĉe granda, 18.900 km³. Oni konsi
          Read more

          Liste der Baudenkmäler in Enneberg

          Image
          Die Liste der Baudenkmäler in Enneberg (ladinisch Mareo , italienisch Marebbe ) enthält die 36 als Baudenkmäler ausgewiesenen Objekte auf dem Gebiet der Gemeinde Enneberg in Südtirol. Basis ist das im Internet einsehbare offizielle Verzeichnis der Baudenkmäler in Südtirol. Dabei kann es sich beispielsweise um Sakralbauten, Wohnhäuser, Bauernhöfe und Adelsansitze handeln. Die Reihenfolge in dieser Liste orientiert sich an der Bezeichnung, alternativ ist sie auch nach der Adresse oder dem Datum der Unterschutzstellung sortierbar. Liste | Foto   Bezeichnung Standort Eintragung Beschreibung Metadaten ja Altes Gasthaus Janesch ID: 14567 46° 42′ 54″ N, 11° 52′ 42″ O 76882557800 0 8. Sep. 1986 (BLR-LAB 4974) Ländliches Wohnhaus/Bauernhof KG:  Welschellen Bauparzelle: 391 ja Asch (Prack) ID: 14540 46° 43′ 12″ N, 11° 54′ 35″ O 75496485300 20. Nov. 1950 (MD) Ansitz KG:  Enneberg Bauparzelle: 38 ja Biccia mit Stade
          Read more