Does mean centering reduces covariance?












2












$begingroup$


It might be a very naive question, but assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal". Does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so thinking it should do the same for covariance.






correlation covariance random-vector







share|cite|improve this question









$endgroup$

















    2












    $begingroup$


    It might be a very naive question, but assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal". Does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so thinking it should do the same for covariance.






    correlation covariance random-vector







    share|cite|improve this question









    $endgroup$















      2












      2








      2


      1



      $begingroup$


      It might be a very naive question, but assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal". Does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so thinking it should do the same for covariance.






      correlation covariance random-vector







      share|cite|improve this question









      $endgroup$




      It might be a very naive question, but assuming I have two non-independent random variables and I want to reduce covariance between them as much as possible without loosing too much "signal". Does mean centering help? I read somewhere that mean centering reduces correlation by a significant factor, so thinking it should do the same for covariance.






      correlation covariance random-vector




      correlation covariance random-vector






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked 2 hours ago









      lvdplvdp

      113




      113






















          1 Answer
          1






          active

          oldest

          votes


















          3












          $begingroup$

          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "65"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f391838%2fdoes-mean-centering-reduces-covariance%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          3












          $begingroup$

          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago
















          3












          $begingroup$

          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago














          3












          3








          3





          $begingroup$

          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.






          share|cite|improve this answer











          $endgroup$



          If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
          $$
          begin{aligned}
          operatorname{Cov}(X + a, Y + b)
          &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \
          &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \
          &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \
          &= E[(X - E[X])(Y - E[Y])] \
          &= operatorname{Cov}(X, Y).
          end{aligned}
          $$
          Centering is the special case $a = -E[X]$ and $b = -E[Y]$, so centering does not affect covariance.




          Also, since correlation is defined as
          $$
          operatorname{Corr}(X, Y)
          = frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          $$
          we can see that
          $$
          begin{aligned}
          operatorname{Corr}(X + a, Y + b)
          &= frac{operatorname{Cov}(X + a, Y + b)}{sqrt{operatorname{Var}(X + a) operatorname{Var}(Y + b)}} \
          &= frac{operatorname{Cov}(X, Y)}{sqrt{operatorname{Var}(X) operatorname{Var}(Y)}},
          end{aligned}
          $$
          so in particular, correlation isn't affected by centering either.




          That was the population version of the story. The sample version is the same: If we use
          $$
          widehat{operatorname{Cov}}(X, Y)
          = frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright)
          $$
          as our estimate of covariance between $X$ and $Y$ from a paired sample $(X_1,Y_1), ldots, (X_n,Y_n)$, then
          $$
          begin{aligned}
          widehat{operatorname{Cov}}(X + a, Y + b)
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n (X_j + a)right)left(Y_i + b - frac{1}{n}sum_{j=1}^n (Y_j + b)right) \
          &= frac{1}{n} sum_{i=1}^n left(X_i + a - frac{1}{n}sum_{j=1}^n X_j + frac{n}{n} aright)left(Y_i + b - frac{1}{n}sum_{j=1}^n Y_j + frac{n}{n} bright) \
          &= frac{1}{n} sum_{i=1}^n left(X_i - frac{1}{n}sum_{j=1}^n X_jright)left(Y_i - frac{1}{n}sum_{j=1}^n Y_jright) \
          &= widehat{operatorname{Cov}}(X, Y)
          end{aligned}
          $$
          for any $a$ and $b$.







          share|cite|improve this answer














          share|cite|improve this answer



          share|cite|improve this answer








          edited 1 hour ago

























          answered 1 hour ago









          Artem MavrinArtem Mavrin

          34516




          34516












          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago


















          • $begingroup$
            Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
            $endgroup$
            – lvdp
            26 secs ago
















          $begingroup$
          Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
          $endgroup$
          – lvdp
          26 secs ago




          $begingroup$
          Marvin, thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduces the covariance?
          $endgroup$
          – lvdp
          26 secs ago


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Cross Validated!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f391838%2fdoes-mean-centering-reduces-covariance%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          Reichsarbeitsdienst

          Image
          Hausflagge des RAD mit Symbol Staatssekretär Konstantin Hierl schreitet auf dem Tempelhofer Feld die Front des RAD ab. Maifeier Berlin, 1. Mai 1934 RAD beim Straßenbau, 1936 RAD-Parade: „Der große Aufmarsch der 38.000 Arbeits­dienst­männer vor dem Führer auf dem Zeppelin­feld“, „Reichs­partei­tag der Arbeit“. Männer des RAD mit ge­schulter­tem Spaten vor Hitler im Auto, 6.–13. Sep. 1937 Reichsparteitag Nürnberg, Arbeits­dienst-Zeltlager mit Arbeitsmaiden, August 1939 Paramilitärischer Aufmarsch des Ar­beits­dienstes, ca. 1940 Vormaliges RAD-Lager (4/224), Ober­mill­statt, um 1940 erbaut als Quartier für junge, zum Arbeits­dienst eingezogene Frau­en, auch Bund Deutscher Mädel (BDM), die bei den Ober­mill­stätter Bauern ein­gesetzt waren, 1950 Der Reichsarbeitsdienst ( RAD ) war eine Organisation im nationalsozialistischen Deutschen Reich. Das Gesetz für den Reichsarbeitsdienst wurde am 26. Juni 1935 erlassen. § 1 (2) lautete: „Alle jungen Deutschen beider...
          Read more

          Tanganjiko

          Image
          Lago Tanganjiko Situo centra Afriko Ĉefaj fontoj Kalambo, Lufuko, Malagarasi kaj Ruzizi Ĉefaj elfluoj Lukugo Setlejoj Bujumbura, Kalemie Datenoj Koordinatoj 6°  S , 30°  O -6.3827777777778 29.615 782 Koordinatoj: 6°  S , 30°  O DEC Supermara alteco 782 m Surfaca areo 32 893 km² f5 Maksimuma longo 673 km f6 Maksimuma larĝo 72 km f7 Akva volumeno 18.900 km³ dep1 f8 Maksimuma profundo 1470 m f10 Averaĝa profundo 570 m f11 v   •   d   •   r Mapo de la lago Tanganjiko estas lago en orienta Afriko, inter Burundo, Kongo Kinŝasa, Tanzanio kaj Zambio. Ĝi apartenas al la Granda Rifto kaj estas unu el la Afrikaj Grandaj Lagoj. Tanzanio (46%) kaj DRK (40%) havas la majoritaton de la lago. Ĝi havas longon de 673 km, areon de ĉ. 32.900 km² kaj maksimuman profundon taksatan je 1470 metroj; pro tiu ĉi granda profundo ankaŭ ĝia volumeno estas sufiĉe granda, 18.900 km³. Oni konsi...
          Read more

          Norda sulo

          Image
          Norda sulo Biologia klasado Regno: Animaloj Animalia Filumo: Ĥorduloj Chordata Klaso: Birdoj Aves Ordo: Pelikanoformaj Pelecaniformes Familio: Suledoj Sulidae Genro: Moro Morus Specio: M. bassanus Dunomo Morus bassanus Linnaeus, 1758 Konserva statuso Konserva statuso: Malplej zorgiga Vivejo de la norda sulo Aliaj Vikimediaj projektoj Sistematiko en Vikispecioj Komunejo pri Norda sulo v   •   d   •   r Uzinte rubaĵojn kiel nestomaterialo. La Norda sulo ( Morus bassanus aŭ Sula bassana ), estas granda marbirdo (ĝis 90 cm, nome la plej granda sulo) de la familio de Suledoj kaj genro Morus , diferencaj de aliaj suloj per pli svelta kaj flaveca kapo. En ties mondoparto ĝi estas nomata ankaŭ simple kiel Sulo . Ĝi estas mara birdo kiu ripozas surtere nur por nestumi kaj idozorgadi; ĝi reproduktiĝas en kolonioj de ĝis 35,000 paroj ambaŭflanke de Norda Atlanti...
          Read more